Why you NEED MFT - "Managed" File Transfer - in this 2020 Decade

(Or, what the "Managed" part of MFT is all about)

Originally written for The MFT Gateway Blog; Apr 24, 2020

Not to sound opportunistic, but the whole COVID-19 situation is leading online businesses to boom like never before (source: BBC). With this, inevitably, the value of computerized trading and automated, secure trade document exchange comes boldly into the picture; and this is also the part where we say, "you should have adopted a MFT solution last year... but it's not yet too late!"

But I already have "File Transfer" - why "Managed"?

Let's take the best path to learning - by example.

Assume you need to send a long list of order items to your vendor. You and your vendor have decided to use the world-famous File Transfer Protocol (FTP) to exchange these docs. It's all very simple:

ftp
open your.partners.host port
cd incoming-orders/your-orders-folder
put your-orders-file-yyyy-MM-dd-HH-mm-ss.csv
exit

Done. The file is transferred.

And now, it's time for the questions.

Burning Questions: for Your Brow

Ten files a day? Fine. Hundred? Thousand?

COVID is snarling, and ironically your business is literally exploding. Now, each day, you have to send dozens of these files, to different FTP servers across your vendors - yes, it recently became plural when you decided to expand your online storefront.

Running a hundred ftp command scripts each day by hand, is not exactly a pleasant experience. Plus, you learned the lesson when you mistakenly uploaded file order-A to partner B's server. So you set up a cool program, and script it to automagically send each file to the correct partner - as soon as you drop them into a monitored folder.

Did the whole file get through - unchanged?

Now the script is doing the job - like a boss. But is it doing the job right? Are you sure?

Say the file was in a CSV (spreadsheet-like) format. Do all the rows get transferred? What if the last hundred rows get dropped during the transfer (say, a network glitch) and nobody notices - not your script, not the ftp command, not your vendor?

Worse... what if a few bits got flipped, and a 10000-crate order got changed into a 90000 one?

Yes, we need to get a file "summary" from the vendor (trading partner) - and ensure it matches with the "summary" on our end. We need to compare the digest (hash; like MD5, SHA-1, SHA-256 etc.) of the final file on both ends, to make sure nothing got corrupted or tampered.

Did somebody else (secretly) tap the transferred file?

Phone taps are still fairly common, and computer network taps are no exception - especially when unprotected, plain-text protocols like FTP are in play.

You could encrypt the traffic by switching to FTP/S or SFTP so that the thief trying to tap your file would get just the encrypted (gibberish) blurb.

But... read along.

Who is at the other end? Your partner, or an imposter?

There are easy techniques (like DNS spoofing) even to trick your system into connecting directly to an attacker's system - especially if you're not careful and attentive enough. You would use end-to-end encryption and stuff, but would be sending the data right into the fox's den.

So we need a mechanism to ensure that only our vendor would be able to make sense of (decrypt) the data we send. Perhaps by encrypting it with a public key whose private key part is owned by him and him alone.

The most fundamental: Did your partner see the file?

Your script happily uploads hundreds of files each day; but does it guarantee that those vendors are even looking at those files? If you don't receive back a confirmation, how can you know if the file got to the wrong system - or got lost in the crowd - or some other nasty thing happened?

Yes, we need an acknowledgement (receipt) from the vendor that he got our file.

Burning Questions, Part 2: for Your Partner's Brow

Sadly, that's not the end of the story; your vendor-partner is even more skeptical - almost paranoid, to say the least.

Was it really that guy (or gal) who sent this file?

The imposter thing could work the other way too. Somebody impersonating you, could send a funny order to your vendor; and the next thing you know is, opening your door to a thousand crates of icky goo.

So your partner needs your unique signature at the bottom of that file - to ensure that it came from you.

Were those numbers in the file, corrupted or tampered while in transit?

Already covered this from your own perspective - digest. Moving on.

Did somebody else tap the file?

Also covered - public-key encryption.

How the heck am I supposed to do anything useful in my day - when I have to keep on decrypting, digesting, verifying and acknowledging dozens of files from that guy (or gal)?

Yes. Your partner needs to seriously think of automating the whole thing. Like you did.

Enter: "Managed" File Transfer!

Say you hired a whole development team, worked day and night, and implemented a solution that covers all of the above concerns:

• adds a signature alongside the file content - so your vendor can verify that it indeed came from you
• encrypts the file using your vendor's public key - so only your vendor can decrypt it, only using his private key
• sends a receipt back when a file is received, so you know for sure the file actually reached your vendor's system
• calculates a digital digest (hash) of the received file and sends it back with the receipt, so you can compare it with your own hash - and make sure the file is intact in your vendor's side
• and most importantly, automates all of these steps and actions and raises red flags if anything fishy goes on

Congratulations!

You just wrote your own "managed" file transfer solution!

But why did you?! MFT is already out there!

If you check out the features of AS2, for example, you would be surprised (or maybe not?) - it already covers all of those concerns you painfully scratched your brow over!

And it already implements everything that your hired dev team had to go through, day and night!

If you check any other protocol (AS4, OFTP, ??, and the like), the observation will be more or less the same. But AS2 is the most sought-after MFT protocol in the business world (heck, Wal-Mart enforces it!), and going with the trend always has its benefits.

So, that's the beauty of Managed File Transfer, MFT.

It covers the end-to-end delivery and acknowledgement/reporting of files; so once you hand over a file to your MFT system, rest assured that:

• either the file will be successfully delivered and acknowledged by your partner's system,
• or (if something goes wrong) your team will be duly notified.

No more staring at ftp folder listings or long dashboards, the whole flow is fully automated and streamlined.

Most MFT platforms also offer value added services (VAS) on top of the basic MFT protocol; such as document (usu. EDI) translation, validation, auto-generation of response documents, and various integrations with third-party systems like CRMs, warehousing and BPMSs.

Meanwhile others give further flexibility by offering intuitive low-level interfacing/integration options; like SFTP- or AWS S3-based triggering of file send-outs, REST APIs for fully automating the management and monitoring of the send/receive process, email-based instant notifications, and so forth.

Okay, back to work - your orders are waiting!

Now, before you go back to uploading those files via FTP, you have a choice to make:

• continue doing business as usual - upload, wait, call the vendor if nothing happens, rinse, and repeat for the next file?
• hire an exorbitant dev team and build your own "managed" file transfer solution?
• install, or more easily, sign up for one of those popular MFT services out there - and dedicate your valuable time to developing your booming business - rather than bothering with file exchanges?

I guess now you know the answer.

Good luck! And we're also waiting eagerly for your queries!