Certain import data files may take more time than others depending on the content and number of items
SyncForce supports a variety of different data format import types to get data into SyncForce and is constantly expanding, improving and evolving. Since we support multiple imports for multiple customers we have a number of Import Processors configured. This number can be scaled up when the volume of import files exceeds a certain threshold.
What can be imported?
We support a number of import data format types for different parts of SyncForce:
- Direct xml upload (via FTP/Dropbox)
- Excel templates (downloaded from SyncForce, updated by the customer and imported via SyncForce interface)
- Excel/csv upload (via FTP/Dropbox)
Import process
When a file is provided, this file will be processed in the following order:
- A file is always placed in a queue for processing. Depending on the amount of import files and number of items in the import files it might take a while before the file is being processed.
- Before processing begins, the file format is validated to ensure compatibility. Depending on the provided format (e.g., CSV, Excel, XML, or import template files) and size (number of items), this validation may take anywhere from less than a second to a few minutes.
- After validation, the file is transformed into one or more import items. If the file contains a single item, this step typically takes milliseconds to a few seconds. For files with multiple items, the transformation may take seconds to minutes. For example, a PDM Master Data template containing 5.000 products and hundreds of attributes may require 15–30 minutes to process.
- Once transformed, the Import Processor validates the items against the customer’s configured platform settings (e.g., attributes, types, languages, markets). The processing time varies per import item, ranging from milliseconds to several minutes.
- Finally, when all import items are processed, the results are aggregated, and the import process is completed and also depending on the number of import items the response to the dashboard might take up to a few minutes.
File formats
XML
Most of the provided XML files are single-product XML files. When uploaded, they are almost instantly scheduled for processing. The actual processing of such an XML file can take 5–10 seconds, depending on its content and complexity.
These XML files are (almost always) translated into a format that the SyncForce Import Processor can understand. If there are many XML files, they are placed in a queue (per customer). If an XML file contains multiple items (e.g. an IMDE-to-SyncForce XML), it can be split into multiple single-item XML files before being processed by the Import Processor.
Product pricing import XML files are handled differently. These imports are processed as a single batch, regardless of the number of products or price regions. This approach improves import speed and prevents scenarios where products are updated and released/published multiple times during the import. Depending on the number of products in the XML file, this process may take 5–15 minutes.
Account pricing imports are handled in two steps: first, updating the existing account, and second, updating all product prices "at once." Similar to product pricing imports, this process may also take 5–15 minutes, depending on the number of products in the XML file.
Import Templates (XLSX)
Several reports available in SyncForce are configured not only to provide a flattened view of products, projects, digital assets, trade hierarchies, etc., but also to facilitate re-importing into SyncForce after customers update the content.
These reports include not just products, digital assets, or other items but also essential configuration details such as attributes, markets, languages, product classes, and digital asset types. These details are necessary to transform the Excel file into viable import items.
Since templates provide a simplified, flattened overview of the content, these files must first be converted to XML format for further processing. For more details, refer to the section on XML format.
Other formats
Other formats like CSV or Excel format are first converted to XML.