Bulk Content Ingestion API
Kaltura offers bulk content ingestion to support scenarios where you need to ingest a significant number of Internet accessible files - from more than a few to many files. Bulk upload presents a great advantage for consolidating large amounts of video content in different locations from remote sites.
You can import multiple files per session using a simple comma separated CSV file, along with their basic metadata fields, or via an XML file, providing complete metadata and setting special configurations per entry.
With these options, you can also ingest files from your own FTP server, or any publicly accessible file server.
Note: For any bulk actions that will create more than 5,000 entries (e.g., users), including categories bulk uploads, please contact your Kaltura representative to coordinate the upload.
Introduction Video
The following video provides a quick walk through of the bulk-upload features in KMC:
Bulk Upload Input Formats (CSV vs. XML)
The benefits of using bulk upload to ingest your files include:
- Importing multiple files in one action, which automates automating the ingestion process
- Ingesting large-sized files that could otherwise take a long time when uploading from a desktop
- Automatically populating metadata fields in the same operation as ingesting the files
There are two methods for bulk-uploading of content:
- The simple method - using a CSV formatted file, which provides a quick and easy way to submit a list of files that you can create in a spreadsheet application like Microsoft Excel or Google Docs. Please note that the CSV format may only be used to upload/ingest new files into Kaltura.
- The robust method - using an XML formatted file, which provides a flexible structure for inputting complex metadata fields and other objects, such as caption files, and allows for more flexible ingestion workflows. The XML format supports full CRUD (Create, Read, Update, and Delete) operations with Kaltura Entries.
How to Submit Bulk Upload Jobs
- Download the sample CSV bulk upload file.
- Review one of the following files:
- CSV - Review the file named: kaltura_batch_upload_falcon.csv in the downloaded zip.
- XML - Review the file named: kaltura_batch_upload_falcon.xml in the downloaded zip.
Bulk Upload with CSV
- To specify the fields and their order in the CSV file, the first line in the CSV file should begin with an ‘*’ (asterisk sign) followed by the list of field names, where each new field is separated by commas.
- Lines with a ‘#’ (hash sign) are treated as comments, and will not be processed.
- Each line after the fields definition line (which begin with an ‘*’ sign) represents a Kaltura Entry to be ingested, and should include the values of every field defined above.
- All entry fields are processed according to the definition line and the order of the fields in the line.
Bulk Operations with XML
The hierarchical structure enabled by XML allows for more flexibility to define a complete content package including all of the objects related to Kaltura Entry, the video source file, base and custom metadata, distribution profiles, transcoding flavors, thumbnails, caption assets, access control profiles, and more.
With XML, you can achieve a simplified integration with other systems (for example, migrating media files including their complete metadata from lecture capture systems).
- In the XML format, the order of the fields is not important (since each field is defined atomically).
- Additionally, fields with an empty value can be skipped and not defined in the entry element.
- The structure and field names should respect that of the Kaltura Bulk Upload XML Schema.
If you’re comfortable using PHP as a programming language, we recommend reviewing the GitHub project: Kaltura’s Bulk Upload XML Content Migration Sample Scripts.
The Bulk Upload Job API
- After you create the CSV or XML file that you want to submit for ingestion, call the media.bulkUploadAdd action to execute the ingestion job.
- Once submitted, you can use the bulk service to review the status of your ingestion job or to cancel it.