During many customer projects, the converter was able to prove its practical suitability on the one hand, and on the other hand, numerous standard functions were created or further developed within the scope of these projects.

Due to its modular plug-in concept, it already provides many functions required in everyday use, but is also open for extensions to solve new problems.

The requirements for the basic use of the converter are deliberately kept low.

Linux OS or a Docker container are sufficient for operation. This makes the Converter ready for integration into modern business infrastructures.
Additionally, the tool is designed and optimized for really big data.
If Big Data is not just a buzzword for you, then the Converter is the right software for you.

The Converter is part of the JANE and LARS software suites from NextStride and fulfills various tasks both within the company and for its customers.

The functions

The functions

NextStride Converter obtains its data from relational database systems, REST interfaces, Elastic Search or CSV files. Data from REST sources can be read in XML, JSON or CSV formats.

The processing of the respective sources is performed by specialized handlers that know how to deal with the source.

In order to combine the source data in a reasonable way, the converter offers various methods to process input data.

These operations do not stop at system boundaries, but are able to combine and process several different source systems and therefore turning data into valuable information.

Plugins take care of processing

The default plugin

The processing of data in the NextStride Converter is controlled by a sophisticated plug-in concept. Depending on the processing, different components come into operation, which can be combined almost arbitrarily.

The default plugin is the centerpiece of the processing. Among other things, it offers the possibility to use field-level JavaScript operations; JavaScript because of its ease of use and its high level of familiarity.

 With the help of the format manager, simple or nested enrichments of structural data, for example the assignment of server names to organizational units, can be implemented.

Additionally, regular expressions and pattern matching can be used here. The support of SAS is based on our mainfame origin.

Another function of the default plugin is the repository manager, which enables the enrichment with historical datasets. It ensures that the enrichment is done exclusively with data that was valid at the time of creation.

 Also included in the scope of the default plugin is a function to harmonize timestamps. This is always useful when the supply of timestamps from different sources happens in different accuracies. In these cases they can be brought into line by means of grouping or rounding.

Finally, the default plugin offers the possibility of a filter function, to prevent unwanted data sets from being processed in the first place.

Other plugins

If the features of the default plugin are not sufficient, specialized plugins are available that can be used to solve specific tasks.

How does the flow control work?

NextStride Converter processes can either be started periodically by scheduler or run as a continuous process that performs processing incrementally with checkpoint handling.

Batch or datastream? You decide.

Data from different sources can be linked at any time using JOIN or UNION operations.

Handling of intermediate results

(Intermediate) results can be stored in NextStride Converter at any time for further processing. An internal data store is available as a target for persisting the results.

Passing on results is not limited to a consumer. Using target handlers, (partial) results can also be passed on to several consumers in parallel for further processing without prior persistence.

… and if it does not work for once?

Error situation

Aborts and error situations can occur during processing for many reasons. Corrupt or incomplete files, an unavailable database connection; the list goes on and on. To ensure that such a situation remains under control and does not lead to considerable manual cleanup activities, the NextStride Converter includes a sophisticated restart procedure and a duplicate key check, which can be used to check as to whether a data set has already been processed or not.

This makes it possible to close data gaps without having to clean up entire areas beforehand.

What happens to the extracted information?

After enrichment and transformation of the data, the extracted information is usually made available for additional processes such as reporting or as input data for further downstream processing steps.

NextStride Converter supports the following sinks:

 Relational database systems using JDBC or „local load function“.

• (Oracle, MySQL, PostGres, DB2, MSSQL,…)
• Elastic Search
• Cassandra (non-relational cloud-based database)
• CSV, via file transfer also directly transferable to remote systems
  (FTP, FTPS, SFTP, CIFS…)

How can the processing be monitored?

NextStride Converter can monitor processing and generate alerts when defined events occur. For example, an alert may be generated when threshold values are exceeded.

Cockpit

Interested users can be informed about such events via email or SNMP trap. However, the notification method is not limited only to the above-mentioned possibilities.

Real-time alerting using web hooks is also supported. Thus, automated reactions to reported events can be implemented.

Even outside of exceptional situations, it may be necessary to inform interested consumers about the progress of processing. The converter supports this for example by generating and sending emails.

What can the scheduler do?

Scheduler

The integrated scheduler enables the definition of job chains with predefined conditions. That means that a job will only start its service when all defined conditions are met. This ensures that the data from all previous systems is available before the actual processing begins.

The scheduler takes over job management, including altering in the event of an error. For further analysis and control, a task log is available for each job.

A web-based GUI was designed for job operating, which guides the operating work in a targeted manner and reduces complexity.

Application examples

The following are some examples of how NextStride Converter is used daily by our customers.

• An insurance company uses the Converter to read out structural data from its in-house configuration management database (CMDB) on a daily basis, thus making core data (e.g. billing information) centrally available for further processing steps.
• A customer from the banking environment uses the Converter to export and import data from a data warehouse. It enables the simple transport of data from A to B.
• For various customers, complex ETL processes with nested JOINs have been implemented using the Converter.
• Also at one banking customer, for performance reasons, in-memory aggregations of up to 250,000 records per minute are performed in the Converter for a real-time dashboard.
• Some customers use the converter to create SAP posting folders for example for accounting tasks and submit them automatically for booking.
• The operating of a banking customer uses the alert function of the Converter to generate SNMP traps in order to be informed about problems at an early stage.
• The results of processing the Converter are now increasingly being used to manage data lakes. Here, interested users can access the data in order to answer their own questions.
• The Converter can also be used for archiving source data.

Why NextStride Converter?

Projects have to be implemented within the conflicting demands of cost, quality and time. That’s why it is important to know what you can rely on. In the following, we will take another look at the key features:

NextStride Converter…

• offers stable automated preparation and availability of information on processes and technical key figures
• is restart-capable and offers duplicate checking
• allows the connection and processing of different sources and transformation of data
• is suitable for integrating legacy systems into modern infrastructures
• supports easy management of ETL processes
• offers expandability through a modular plugin concept (source handlers, converter plugins and target handlers)
• offers easy usability through a web-based user interface
• is suitable for use in cloud environments
• offers mature and reliable technology with high usage by customers and leads to high customer satisfaction!
• is optimized for big data

What conditions must be met?

To ensure that NextStride Converter can be used in your environments smoothly, we have made sure to keep the requirements as low as possible:

Running NextStride Converter only requires a Linux OS or Docker container,
a Java runtime and the availability of resources depending on the amount of data and functions used.

We will happily support you with the sizing!

Schematic representation