“It’s not just what you write, but how you write it.”
– Wilma Davidson
Tecplot 360 performance improved significantly in the last five years. However, the performance a user is experiencing varies dramatically with the file format being read. Whenever we tell one of our software developers that a customer is experiencing poor performance, their first question is always “What data format are they reading?” I’m writing this blog to explain why.
Tecplot File Formats
» Watch this video: Tecplot Data File Types
» Learn more about SZL Technology
There are three main Tecplot formats that are in use today:
- SZL or Subzone loading format (with extension .szplt) is the newest.
- PLT is the legacy binary file format (extension .plt).
- DAT is an ASCII version of the PLT format (extension .dat).
I thought it would be useful to compare the file size and Tecplot load times of these file types for a common CFD use case – the NASA Trapezoidal wing.
The NASA Trapazoidal wing case has 204 million finite-element cells in the volume zone, and another 5.1 million surface cells to define the geometry. The surface cells are stored in a separate PLT file which is read in at the same time as the SZL, PLT, or DAT file. The plot is the geometry of the aircraft and the Cp = -2.0 isosurface.
Note that the time measurements include both reading the file and generating the image. Both SZL and, to a lesser extent, PLT delay the loading of data until you select an object (slice, isosurface, or streamtraces) to display. The “read” operation only reads the file headers. This also means the time to read the data can vary substantially with the type of plot you are doing, and the timings given here are for the specific plot that I have created – an isosurface of Cp = -2.0.
Whenever you measure performance you need to take disk caching into account. When you read a file the operating system will save selected portions of the file in a cache so that subsequent reads of the file are faster. Caching occurs on the local computer and, if you are reading from a network drive, on the remote file server. To get consistent timings, I used the local hard disk in my engineering (MS Windows) workstation and followed a consistent process:
- Shut the computer down and turn it back on to completely clear the cache for the uncached measurements, and
- Load the file several times to get a fully cached measurement.
The following table contains the results.
|File Size (GB)||6.19||9.95||30.3|
|Load Time (sec) – uncached||21.7||88.6||8101|
|Load Time (sec) – cached||7.3||46.8||8008|
File Size Comparison
Note that the file sizes are very different for the three file types, with the SZL file being 62% of the PLT file and just 20% of the DAT file. This DAT file was created by reading the PLT file into Tecplot and writing out as an ASCII file using the default settings. By default, it writes 9 significant digits for each of the floating-point variables. The DAT file size could have been reduced somewhat by reducing the number of significant digits, but that is risky for viscous flow grids which are highly stretched to resolve the boundary-layer near the wall.
SZL is the fastest file format by a factor of 6.1 to 6.4 over PLT and by a factor of 373 to 1097 over DAT. This is to be expected because, when loading SZL files, only the data near the isosurface (and the airplane surface) are loaded. For PLT, the coordinate (X, Y, Z) and isosurface (Cp) variables are loaded for the entire volume zone and for DAT all variables are loaded multiple times (more on this later). DAT was the slowest format, by a factor of 373 to 1097 over SZL and a factor of 91 to 171 over PLT.
Clearly, you want to avoid loading large DAT (ASCII) files whenever possible!
Another interesting thing is that the DAT format had very little benefit from disk caching.
So, why is DAT (ASCII) so slow? When loading a DAT file, Tecplot first translates it into a PLT file using the equivalent of the Preplot utility, and then it reads the PLT file. The process is as follows:
- Read the DAT file and write the header information into one binary file and the zone data into a second binary file.
- Read the binary header and zone data files and write them out as a PLT file.
- Read the PLT file (headers, coordinate variables, and Cp variable).
Not only is the larger size of the DAT (ASCII) file making the initial read much slower, but the data is then written, read, written, and read again before the plot can be generated. This is a lot of reading and writing!
Obviously, you should avoid reading large DAT (ASCII) files whenever possible. Here are some specific recommendations:
- If you are a code developer, write binary (SZL or PLT) files directly using our free TecIO Library.
- If you are using a code that writes DAT (ASCII) files, and you can’t modify it to write the binary files using Tecio, convert the files to binary (SZL, or PLT if it contains polyhedral cells) before you read them.
- For SZL, read the file into Tecplot in batch mode and write it out as SZL.
- For PLT, use the preplot utility.
Converting DAT (ASCII) files to binary will still be slow, but it can be done in batch mode so that you aren’t sitting there twiddling your thumbs while it converts. It also has the benefit of compressing the files so that you use less disk space!
Finally, some codes write DAT (ASCII) files but give them a *.plt extension. Tecplot will recognize that these files are ASCII and will go through the translation process just as it would it if was an *.dat file. If you suspect this is happening to you, try viewing the file with ASCII file editor (or “more” on a linux platform). It will be obvious if it is a binary file (lots of strange characters).
Stay tuned for a similar blog that I plan to write on the differences between these file formats for time-dependent data!
Contact Technical Support at firstname.lastname@example.org with questions about TecIO or converting your data.
Chief Technical Officer