Fast, Secure Remote Data Access – Tecplot SZL Server

Description

Scott Imlay: We will be talking about our SZL Server for fast and secure remote data visualization. This is our client server implementation. My name is Scott Imlay. I’m the Chief Technology Officer at Tecplot.

The expert today is Dr. Dave Taflin. He is a Senior Software Developer, and did most of the development for the SZL Server. So he is the guy in the know and will be answering our questions today.

What is SZL Server?

Dave Taflin: As you mentioned, SZL Server is our client server architecture for loading data that resides on remote machines. It consists of two components, first off an add-on, and the main dialog to the add-on is shown on the right side of the screen there. And secondly, a remote server that this add-on talks to, to retrieve data from the remote machine.

Now, I should clarify, a server may be a little bit of a misnomer here. Most people think of servers as programs that are running all the time, listening for inbound connections, like a web server. That’s not what we’re talking about here.

Our server is launched for a specific data load, and it exits when the data that it has loaded is no longer needed, such as when you select new layout in Tecplot 360. The remote server may be launched directly by the client, or it may be manually launched by the user, and we’ll go over both of those scenarios during this presentation.

The server, once it’s running, loads data as directed by the client and passes just the subzones that are required to the client on demand for whatever the current data requirements are for the current plot.

And I should specify here that the SZL Server requires our new SZL file format, which has been under development here at Tecplot for the last three years or so. It enables Tecplot 360 to load small subsets of your data, thereby greatly reducing the disc access for the plots that you need to make. Finally, for security we use the SSH encryption and authentication methods.

Scott Imlay: If you are not familiar with SZL, and I know not everyone is, we actually have done some webinars on this that you can refer back to, as well as blogs. So there’s plenty of resources on our website.

Learn more about SZL

Running SZL Server

Dave Taflin: Now I’m going to give a brief demo of one mode of operation for our client server.

First off, under the file menu, you select load remote data. That displays the dialog that you saw on the slide a minute ago. You have three different connections options here. We’re going to pick the default one, SSH tunneling. This, in addition to using SSH authentication, will also encrypt your data for security. My remote machine is named TPB045, and my login ID on that machine is Dave.

I have options here to use an SSH key or SSH-agent. We’re not going to do either of those right now because those require some setup. If you don’t have any setup at all and you’ve just been using your password, the SSH to the remote machine, that’s fine. Just click, “Do not use SSH key,” and go ahead and connect, and you’ll be prompted for your password.

As soon as I click okay here, I’ll authenticate to the remote SSH server, and the SZL server will be launched on the remote machine and connect back to the client here. And then I’ll have a couple more controls I can use as soon as that connection is established back.

The select files button is enabled now. This is allowing me to browse for files on the remote machine. It turns out that since I’ve practiced this, the file that I want to load is already preloaded, and that’s keyed off the remote machine name. So I’m going to go ahead and just click okay here, to load crutcher.szplt.

Now, a question I’ll address later in a little more detail is macros and layouts, but generally speaking these are supported, so I’m going to go ahead and save a layout and then reload it to show you what that looks like. I’ll just name it Untitled and save it, and then go new layout. Now, as I mentioned a minute ago, since I went new layout, the remote server has just exited. But I can load that layout that I just saved. I’ll be prompted for my password again.

And now SZL Server is being relaunched on machine TBP045. As soon as it launches, it connects back and the load data command is issued, and we get the same layout that we had before. So that is our first demo. Let’s move on to some more questions.

How is SZL Server different from remote display?

Dave Taflin: What we usually mean with remote display, is that you’re running Tecplot 360 on the remote machine over in the supercomputer center or whatever, and using the magic of the X-Window system, you’re piping the GUI and all of the graphics in the X-Window form to your local machine.

We’ve seen a lot of performance issues with this, and they’re prevalent because there’s a lot of chatter. The X11 protocol has a lot of chatter. So when you get higher network latencies, the performance degrades significantly, so we’ve seen a lot of performance problems.

Also, of course, in order to run in that way, you have to have Tecplot 360 installed on the remote machine. And although that generally is the case on the larger supercomputer sites, I wouldn’t assume that it’s installed everywhere just yet. We would be happy if it were, however. But the nice thing about SZL Server is that it gives you an option to remotely view your data without having Tecplot 360 installed on the remote machine.

Scott Imlay: And even if it is installed on the remote machine, you often have little control over which version is installed remotely. You may be a couple, three versions behind, which may or may not be a problem, but it’s another issue.

What is needed for the remote server?

Dave Taflin: We currently support running the SZL Server only on Linux machines, so the remote computer has to have Linux installed. Since we’re reliant on SZL technology to limit the amount of data required, you don’t need very much computer power. The remote machine, of course, does need direct access to your data files. But since the SZL Server does not require a Tecplot license, you don’t need to worry about any licensing on the remote machine (only on your local machine). And furthermore, you can install SZL Server under your own account, so you don’t need any administrative permissions.

And then finally, of course, you have to be able to access the remote machine, which you will normally do through SSH, and that is the mechanism that SZL Server itself uses. You can get around that using the manual connection method that we’ll talk about a little more later, but generally speaking, SSH seems to be the standard, and that is the technology that we use by default with SZL Server.

What if my data is not in SZL format (*.szplt)?

Dave Taflin: Well, you have a couple of options. You can use Tecplot 360 as a data converter. Now, this does require that Tecplot 360 be installed on the same machine or a visible machine on the same side of the connection where your data files reside. But to use Tecplot 360 as a data converter, you don’t need a license. So you can, again, just install it under your own account, not worry about any licensing, and then run Tecplot 360 with the -convert flag to convert any data files that Tecplot 360 can load into SZPLT files.

This may be as easy as just a single command line command, but there are some supported data file formats that you can’t load directly from the command line, or you may have non-standard file extensions on your data files. And in those cases, you can create a macro that reads the file and exports it an SZPLT file. You can then run that in batch mode to get the same effect.

The second option, and it’s one that we encourage for those of you who are writing your own simulation codes, is to support writing SZPLT files directly from your application. And you can do this pretty easily using the TecIO library that’s freely available from our website.

Learn more about TecIO

Scott Imlay: And just a little more encouragement for TecIO. The SZL files have some other advantages. They load much faster into Tecplot 360, especially for finite-element data. They’re smaller, especially for finite-element data, and they use less memory. So there’s several advantages in addition to being compatible with the SZL Server.

How fast is SZL Server?

Dave Taflin: Well, in true consultant mode, I should say, “That depends.” Of course it does. It’s impossible to get around certain things that it depends on, particularly the network speed and latency of the network that you’re connecting over.

But generally speaking, for actual remote data, not just something that’s on a different machine on your LAN, but something that’s out on a supercomputer center or up in the Amazon Cloud or something like that, it’s much faster.

A complex plot will take about five times as long to display as if you were loading it from your local hard disk. But that is still an order of magnitude (or two) faster than downloading the file to your local hard disk. So it can save you a lot of time.

Does SZL Server support all data types and zone types?

Dave Taflin: SZL Server supports everything that Tecplot 360 supports, except polyhedral zones. We are working on extending our SZL technology to polyhedral zones. That remains a work in progress.

Are layout files and macros supported by SZL Server?

Dave Taflin: Yes. As I demonstrated a minute ago, if you’ve loaded data via client server, you can save layouts or you can be recording a macro while you’re doing that. And when you play those back, you’ll be prompted for any authentication information that’s required by the SSH server.

There is one caveat to that. On Windows, if you try to run in batch mode and user input is required, like a password or something, that’s going to fail because of the difficulty of maintaining a connection between Tecplot 360 and any kind of a console. Windows likes to launch things and separate them from their terminal window, whereas on Linux or Mac that connection is maintained. So with that caveat, yes, macros and layouts are supported by client server.

And as a side note, remember that layouts and macros are both something that happen on the client side. You sitting in front of your Macintosh or Windows machine or whatever, you’re going to be using Tecplot 360 to run the macro or to load the layout. And from there, all the client server magic happens behind the scenes to load the data from the remote machine.

And you can share layouts and macros as long as the remote machine is still visible, and whatever user that’s running your macros has whatever credentials are required to log in to your account on the remote machine.

Scott Imlay: Layouts and macros are both ASCII files, so you can just email them to people if you want to – they are easy to share.

What if I perform an action that requires me to save my data when I save the layout?

Dave Taflin: This is something like slice extraction, and if you’re an experienced Tecplot 360 user you’ll probably already be familiar with which operations you perform that will require you to save a new data file when you save a layout. It’s a shrinking list, but there are still a few operations that cannot be journaled. Therefore, if you want to save a layout, you’ll have to save a new data file as well. If you go ahead and save a layout in that situation, then your entire data file is going to be downloaded to your local machine.

Probably this is going to be really slow because the sweet spot for client server here is large remote data files. You’ll know that this is happening because you’ll see the dialog at the bottom of the screen here. My recommendation would just be for you to click no and not save the layout. You can still save a macro that performs the same data alterations that brought this situation upon you, but I would not save a new data file. That’s likely to be a very slow operation.

Can I load both local and remote data in a single Tecplot 360 session?

Dave Taflin: Yes, you can! You can do this in a couple of ways.

You can use different frames for each of your individual data loads, just like you’ve always been able to with Tecplot 360.

You can also, if you’ve already loaded remote data into a frame, you can also append data with any loader that supports data appending. The SZL client currently doesn’t support appending, so you can’t append one remote data load to an existing data set yet. That’s in our suggested feature list.

I think the most common scenario is that you’ll want to be comparing two different data sets, possibly from different servers, and you can load those into different frames side by side for comparison.

How secure is SZL Server?

Dave Taflin: It is as secure as SSH itself is, which uses pretty state-of-the-art authentication and encryption methods. As I mentioned before, it uses SSH tunneling to encrypt your data for transmission. We also support multi-factor authentication to the remote SSH server. This is typically how security is enforced for the large supercomputer centers.

Can SZL Server support multi-hop SSH?

Dave Taflin: As I just mentioned, supercomputer centers will commonly have multi-factor authentication. You’ll be, at that point, logged in into what they may call a bastion server, which is kind of the front-end access. It’s like the front door to the supercomputer center. You have to go through that in order to subsequently access the machine that you’re actually interested in accessing. So you SSH to the bastion server, and then from there you can SSH to whatever machine you really want.

And yes, we do support that with the manual connection method, and that’s going to be our second demonstration.

Manual Connection Method Demo

More information is also available in the Manual Connection Mode section of the Tecplot 360 User Manual in our Documentation.

I’ve pulled up an extern window on a Linux machine we have here. I could do this from Windows using PuTTY and PuTTY’s SSH-agent, but it’s a little easier to see what’s going on with the Linux command lines, so I’m going to do this all on Linux.

First off, I’m going to launch the SSH-agent. This allows me to forward some credentials to both of the remote machines I’m going to be accessing. So I’m going to go ahead and launch SSH-agent the usual way. If you use SSH-agent, you’re probably familiar with that syntax. Then I’ll just add my default encryption key to the agent. Now it’s available for forwarding.

The next thing I’m going to do is log into the bastion server, and I’m going to be using remote port forwarding to provide the tunnel for our data to come back to us. So that’s going to look like this. I’ll use the -A command to use the SSH-agent for part of my authentication. There will be more required in a minute, as you’ll see. And -R for a reverse port forward. And then I’m going to pick a port on the remote machine that I know it is available.

You’re supposed to be able to enter 0 and let the SSH server pick a remote port for you. I think there’s a bug in several versions of OpenSSH, including the one that I’m using here, that actually prevents that from working. So you unfortunately have to know an open port number on the remote machine. I’ve had good luck so far with 45678, so let’s see if that keeps working.

And then that’s going to be forwarded to the local host, and I forgot a step, so let me back up and launch Tecplot 360 first, because I need to know what the local port number is. Okay, so I’m going to launch 360 and initiate a remote data load, and I’m going to use the manual connection here and just click connect. So this -P here, that tells me what my local port number is that the SZL client is listening on.

Now I can go ahead and do my SSH to the bastion server first. So that’s going to be SSH -A -R. The remote port is 45678, and that’s going to be forwarded to my local host here, and port 48689. That’ll be different every time you initiate a remote data load.

And then, of course, I have to enter where I’m going and who I am. This is up in Amazon’s Cloud, so it’s going to be ec2-user@ … and this is the IP address of my remote machine that I’ve set up as the bastion server.

And since we have multi-factor authentication going here, I’m using a Google authenticator, so I’ll just enter the six-digit code that’s currently up. And now I’m logged into my bastion server.

Now I turn around and do the same thing again to access the data server that I’m interested in, so that’s going to be another SSH with another reverse port forward. I’m using SSH-agent again to forward my SSH key to the data server.

And then I’m going to use 45678 on the second machine as well, so that’s going to look like this. That will forward port 45678 on the remote machine to my bastion server’s port 45678, which will turn around and forward it back to the machine on which Tecplot 360 is running. And again, I’m ec2-user, and the data server’s IP address is 52.35.147.24.

Because we are using port forwarding, you will need to modify the Manual Connection Mode instruction in the Tecplot 360 User Manual in our Documentation.

Launching SZL Server

First note, I do have SZL Server installed, and I’ve added it to my path, which is required. So I can just type szlserver, and instead of the IP address of my local machine here at Tecplot, I’ll just use localhost, and for the port, I’ll use the port that I forwarded from the remote data server. But I still do need this authentication key, and this likewise, it’s going to be different every time you do a remote data load.

As soon as I launch this, it should connect through all of the port forwarding, all the way back to my local machine here, to the SZL client that’s waiting for it, and indeed it did.

You can see, I’ve also got crutcher.szplt on this remote machine, so I’ll go ahead and load that.

Now I have an oddity with the X11 server that’s running on my Windows machine. It throws some kind of strange event that trips Tecplot 360 up momentarily, but all I have to do is select that 3D plot, and here are again with our Crutcher data. Okay, so that is the demonstration of multi-hop SSH using reverse port forwarding to support multi-hop SSH, using the manual connection method.

As soon as I launch this, it should connect through all of the port forwarding, all the way back to my local machine here, to the SZL client that’s waiting for it, and indeed it did.

Scott Imlay: If you are familiar with how this works, and you have recommendations on making it easier, because that could take a little bit of time, we’d love to hear your recommendations. We’ll have an email at the end of this presentation, you can send any suggestions or questions to.

Send questions or suggestions to Tecplot Support.

How do you get SZL Server?

Dave Taflin: The Linux installer for SZL Server is shipped and installed with every version of Tecplot 360. It’s stored in the SZL Server folder, which is just beneath the Tecplot 360 install directory. You’ll need to transfer it to your remote machine, probably using scp, and then run it on the remote machine. You then have to add SZL Server’s bin folder to your PATH environment variable. You’ll do that by modifying .bashrc and .bash_profile to modify your PATH environment variable. Having done that, you’re pretty much done because there’s no licensing required on the remote machine.

Can SZL Server work in parallel?

Dave Taflin: Each instance of SZL Server will use multi-threading, but it is not running in MPI parallel, if that was the intent of the question. However, each remote load that you do is going to launch a different instance of SZL Server, so you can get some parallelism there.

Dave Taflin: If you are familiar with Amazon ec2, I typically use one of the smallest available machines as a server, which has only a fraction of a gigabyte of memory. And I’ve been able to load most of our example data files, which can run into at least the hundreds of megabytes. Because, as Scott said, with the end of the two-thirds scaling for dropping a slice, you wind up not loading all that much data.

Thank you for watching the Webinar!

Scott Imlay: Thank you all for joining us today. We recorded this webinar, and we’ll have it available for you on our website. And you can get more information about SZL Server also at our website, and of course you can send an email to Tecplot Support to ask additional questions. Thank you again. Thank you, Dave.

Dave Taflin: You are welcome, and happy Tecplotting, all.

Try Tecplot 360 for Free

Related Videos