The functionality of FlowHeater closely resembles the concept of a continuous-flow water heater. A wide variety of data sources can be interconnected with FlowHeater and by making use of exchangeable Adapters there is no restriction to possible data flow and conversion.

For example, the data in Oracle database tables can be copied directly and without manual intervention into an MS SQL Server database. And while copying proceeds the data can be "heated", i.e. converted, amended, transformed, manipulated and additional dynamic or static data injected, as predetermined by a Definition designed with the help of an intuitive user interface.

Adapter TextFile
The TextFile Adapter is used for the import and export of text data. The Adapter can be assigned to both READ and WRITE sides of a conversion. If you want to read or write CSV, TXT, ASC, ASCII, flat files or other text files and convert or transform these into any other format, the TextFile Adapter is perfect for the job. Please also refer to the general information about the usage of Adapters.

Of all the text file formats, CSV (Comma Separated Value) is probably the most popular. The FlowHeater’s TextFile Adapter comprehensively supports CSV files and practically any tabulation of data in text form as well as conversions of the most varied mixtures of character encodings (codepages). The default encoding is the codepage currently configured for your operating system, but alternative codepages can be specified and the actual selection is stored along with the Definition. For example, a Definition could have been created on a German workstation, but can later be executed without causing any problems using the batch module on a Windows Server, where the system codepage is configured for English. Support for DOS (ASCII), Unicode (utf7, utf8, utf16, utf32) as well as EBCDIC (IBM mainframe) is included. The encoding expected for input or chosen for output is simply selected in the "code page" dropdown. On the READ side when selecting a UTF/Unicode file with BOM (byte order mark) the correct code page will be recognized automatically.

Adapter settings in the General tab
Filename:
Choose a CSV text file
Choose a CSV text file
An absolute or relative path to the file to be read or written. Relative paths are always relative to the folder the Definition file is saved in. Clicking on the “…” button at the end of the filename field opens the familiar operating system file selection window, where you can choose a file to open or navigate to the folder you wish to save the file in.
There is no support for wildcards such as *.csv here. If you want to process several text files in a folder you need to use the Batch Module with the option “/ReadFileName *.csv”.
On the READ side http:// or https:// for the resource location of a file can be used. The file must be openly available; presently no user login to a web server is supported.

The first row contains column headings: This checkbox allows you to select whether the first line starts with data or contains field names. If ticked on the READ side, the field names are automatically extracted from the text file and become the default field names for the Definition. On the WRITE side, this option confirms that the field names as specified in the Definition are to be written as the first output row.

Dynamically allocate columns by field names: This instructs the TextFile Adapter to allocate the correct fields dynamically according to the CSV column headings found in the text file.

Codepage: Selects the character encoding to assume or generate.
e.g.
850 = MS DOS ASCII Western European
1252 = Windows ANIS Western European
65001 = Unicode UTF-8
...
Note: READ and WRITE sides can be assigned different codepages.

Delimitation: With these radio buttons you select how the fields in your text file or flat file are divided from one another. The two options are With delimiters and Fixed width. If you opt to use delimiters, you select which one to use on the Delimiter tab (see below). If you switch back and forth between the two options, the default fixed width of 10 may become set. This can be reset to zero by using the Reset field widths button.

Line break: Here you can choose between “DOS / Windows ”, “UNIX / Linux” or alternatively “User defined terminator”. These terms mean:

DOS / Windows: When reading or writing text files of this type, each line is terminated by a sequence of two bytes “CR LF (0x0D0A)”.
UNIX / Linux: Lines are terminated by just a single byte “LF (0x0A)” in this format.
User-defined terminator: This gives you the option to enter any desired character (or string of characters) to be used for line breaks in the text box alongside. The TextFile Adapter will treat these characters as line breaks on reading or writing.
The option "Terminate final record/row with a line break" instructs the TextFile Adapter to ensure that the last row written to the file is followed by a line break.

Blank lines in the text file are ignored if the "Skip empty lines" option is checked. The defined CSV separator, tab or other white space characters contained in the column data are also ignored. That means that a row consisting purely of CSV separator characters will also be filtered out.

Adapter settings in the Delimiters tab
Delimiter: For a CSV file you specify here which delimiter separates the fields being read or should be output to separate the fields written. The standard delimiter is the semi-colon character. If none of the alternatives offered here apply, you can select the "other" option and enter a special character or string in the text box beside it.

Quotation marks: It is common in CSV files, but also in flat files with fixed column sizes, that fields containing text are surrounded by quote marks . The standard is without quotation marks.

123;"Text fields can contain anything; including a delimiter";€100.51;...

You can choose from a selection of characters that are popular for quotation marks here, or define your own special character. Note: If you also selected a fixed width, then the two quotation characters are included in the field’s maximum length. For example, in a field of maximum 20 characters, only 18 characters are left for data.

Only apply to STRING data types: This option tells FlowHeater to only use the specified quotation marks for fields of STRING data type.

Also use for column headings: If you select this option, the settings for field quotation marks also apply for column headings.

Ignore empty fields: If this option is selected, then the quotation marks will not be output for empty fields. This means that empty fields are simply indicated by two consecutive delimiters, making the output more compact.

Check if column delimiter occurs within quoted text: When this option is selected (default) determines whether or not a CSV column delimiter character may appear inside quotation marks.

Automatically replace doubled string delimiter characters embedded in field contents with one character: When the chosen string delimiter character can occur as a normal character in a CSV field content these are normally masked by the CSV string delimiter character simply being repeated. When this option is checked, the TextFile Adapter will collapse these duplicated characters while reading - in other words two consecutive CSV string delimiter characters will automatically become a single character.

When this option is used on the WRITE side, any string delimiter character encountered within the CSV field content will be automatically masked, meaning it is doubled/quoted - in other words a single CSV string delimiter character in the data will automatically become two.

Adapter settings in the Field / Data Types tab
In this tab you can adjust the fields that are considered a part of the data; ignoring or removing fields and inserting new fields / columns and amend the assignment of fields types. The input again from READ Adapter button fetches the column names afresh from the text file. This option is only available on the READ side.
Simply click on a field name in the fields list to select the field to be moved or amended; this highlights the field in this list and its details appear in the Field name and Format sections.

The sequence of processing the currently selected field can be amended here quite easily, by using the Up and Down buttons.

In the Field name text box, the heading assigned to the currently selected field can be amended.

Format: In this section you can specify the field Length and Alignment (left or right justified ). Fixed field widths can either be adjusted explicitly in the Length box or graphically by using the left mouse button to drag the triangular tab stop markers along the ruler (circled here in red). The Pad character is used to define which character is used to fill out the remaining space in any fixed length field that is shorter than its maximum width (left or right justified). The standard character used is a space. Note that this only applies to fixed length fields.

Header / footer rows tab
Using this tab you control the processing of additionally available or to be output CSV header and footer row(s). The processing of header/footer rows does not operate along with the defined data columns, rather for entire rows. Each header/footer row is thereby represented by a separate Parameter.

On the READ side: The complete row(s) are available in predefined Parameters for further processing. Extracting these values during processing is achieved using the Get Parameter Heater.Note: The content of footer row(s) is also read prior to data processing and so is already available for processing by the time the first data row is read.

On the WRITE side: The value contained in the specified Parameter is written to the defined row. The contents of Parameters can be set or modified during the run using the Set Parameter Heater.

The option "Write header rows as a final step in processing" enables you to determine that the contents of the defined Parameters the header row(s) are only written to the CSV file after completion of processing. For example, this enables the results of calculations to be included in header row(s) that are accumulated from the data during processing and only written to the top of the CSV file as a final step.

Settings in the Advanced tab
TextFile Adapter - Advanced CSV settings
TextFile Adapter - Advanced CSV settings
Automatically strip spaces from start and end of field dаta: If you select this option any leading or trailing space characters are automatically removed from each field/column.

Allow line breaks in columns (reading only): With this option you can define whether line breaks are permitted in the field content. You can use this to read, for example, a CSV file that contains line breaks without it leading to column or field displacement. This option is only usable on the READ side.

Automatically replace line breaks with: This option allows you to specify a substitute character that is used for the automatic replacement for line breaks. This is useful when data from an SQL database is exported, for example, that has embedded line breaks. All line breaks in the field contents are replaced.

Maximum length of each row: This informs the TextFile Adapter the longest permitted line size.

Suppress writing empty header and footer rows: With this option you control whether to suppress empty header or footer rows when otherwise selected for output.

FlowHeater: Your specialist for data migration
FlowHeater offers you the perfect solution for transferring data from one database to another. During this process, data from different sources is merged via the FlowHeater adapter. The adapter works as intermediary between data from different sources that are saved in differing formats. And the name FlowHeater says it all: Just like water in a flow heater, data is continuously transformed in the course of one flow cycle as it connects the different data sources via an adapter.

This is how data migration works
Data migration describes the process of transferring data between different systems such as data import and export between different database systems. In most cases, the necessity of data migration arises due to company restructurings or company mergers. Further reasons are updates to newer versions of the existing databases. Data migration is also a great opportunity to clean your data and thus only transfer necessary information to the new system. FlowHeater adapters make sure data migration is quick, clean and safe. It is also important to migrate all data, not just master data. This way, a fast migration is possible and you can start extensive reporting right away.

Advantages of precise data migration
Data migration offers a multitude of advantages, above all, a systematic data clean-up. Data migration is never a topic purely for the IT department but requires inter-divisional cooperation between all departments of a company. It typically proceeds in the following steps: A status analysis and planning are followed by diagnosing possible weak points. After that, which data is to be migrated to where is identified. Based on that step the migration plan is developed.

Technical possibilities
FlowHeater offers you a tool that is much more intuitive than comparable tools on the market. It transfers and converts your data quickly from one system to another. Without time-consuming installation you unpack the zip-folder, without needing any programming skills. The application works as follows: the Read-file is read by the adapter into an internal format. After this step, the fitter transfers the data via pipes and heaters into the internal data structure on the write-site. All data conversions are carried out by the fitter, including ones that fall under different locales such as data formats. Different coding pages also pose no problem for the FlowHeater and can be combined at wish.

Discover our different product solutions
We offer you many different adapters for data migration and guarantee uncomplicated and quick solutions. You can easily combine the different adapters such as adapters for text files like CSV, TXT, ASCII or FlatFile, or for the import and export of data to Microsoft Access MDB, MDE, ACCDB, ACCDE databases. All versions from 1997 to 2016 are supported. The MS Excel adapter guarantees a fast import of your spreadsheet data. Importing databases is especially important: The different adapters for migration database programs such as MySQL, MS SQL, Oracle, SQLite3 and PostgreSQL transfer data reliably and fast.

Only for V.I.P