Monday, December 11, 2017

Fun with Mendelson AS2: Automating your AS2 Workflows

Mendelson AS2 is one of the widely used AS2 clients, and is also the unofficial AS2 testing tool that we use here at AdroitLogic (besides OpenAS2 etc.).

Mendelson AS2

While Mendelson does offer quite a lucrative handful of features, we needed more flexibility in order to integrate it into our testing cycles—especially when it comes to programmatic test automation of our AS2Gateway.

AS2Gateway

A spark of hope

If you have a curious eye, you might already have glimpsed the following on the log window of the Mendelson UI, right after it is fired up:

[8:30:42 AM] Client connected to localhost/127.0.0.1:1235
[8:30:44 AM] Logged in as user "admin"

So there's probably a server-client distinction among Mendelson's numerous components; a server that handles AS2 communication, and a client that authenticates to it and provides the necessary instructions.

The fact is confirmed by the docs.

What if...

So what if we can manipulate the client component of Mendelson AS2, and use it to programmatically perform AS2 operations: like sending and checking received messages under different, programmatically configured partner and local station configurations?

Guess what? That's totally possible.

Mendelson comes bundled with a wide range of Java clients, in addition to the GUI client that you see everyday. Different ones are available for different tasks, such as configuration, general commands, file transfers, etc. It's just a matter of picking and choosing the matching set of clients and request/response pairs, and wiring them together to compose the flow you want.

Which could turn out to be harder than you think, due to the lack of decent client documentation (at least for the stuff I searched for).

Digging for the gold

Fortunately the source is available online, so you could just download and extract it, plug it into an IDE like IntelliJ or Eclipse, and start hunting for classes with suspicious names, e.g. those having "client", "request" or "message" in their class or package names. If your IDE supports class decompilation, you could also simply add the main AS2 JAR (<Mendelson installation root>/as2.jar) to your project's build path (although I cannot guarantee the legality of such a move!)

Well, my understandings may not be perfect, but this is what my findings revealed regarding tapping into Mendelson's AS2 client ecosystem:

  1. You start by creating a de.mendelson.util.clientserver.BaseClient derivative of the required type, providing either a host-port-user-password combination for a server (which we already have, when running the UI; usually configurable at <Mendelson installation root>/passwd), or another pre-initialized BaseClient instance.
  2. You compose a request entity, picking one out of the wide range of request-response classes deriving from de.mendelson.util.clientserver.messages.ClientServerMessage (yup, I too wished the base class were <something>Request; looks a bit clumsy, but gotta live with it—at least the actual concrete class name ends with "Request"!).
  3. Now you submit the request entity to one of the sender methods of your client (such as sendSync()), and get hold of the response, another ClientServerMessage instance (with a name ending with, you guessed it, "Response").
  4. You now consult the response entity to see if the operation succeeded (e.g. response.getException() != null) and to retrieve what you were looking for, in case it was a query.

While it sounds simple, some operations such as sending messages and browsing through old messages requires a bit of insight into how the gears interlock.

Your first move

Let's start by creating a client for sending our commands to the server:

/*
 "NoOpClientSessionHandlerCallback" is a bare-bones implementation of
 de.mendelson.util.clientserver.ClientSessionHandlerCallback;
 you could also use one of the existing implementations, like "AnonymousTextClient"
*/

BaseClient client = new BaseClient(new NoOpClientSessionHandlerCallback(logger));
if (!client.connect(new InetSocketAddress(host, port), 1000) ||
        client.login("admin", "admin".toCharArray(), AS2ServerVersion.getFullProductName())
                .getState() != LoginState.STATE_AUTHENTICATION_SUCCESS) {
    throw new IllegalStateException("Login failed");
}
// done!

My partners!

For most of the operations, you need to possess in advance, Partner entities representing the list of configured partners (and local stations; by the way, I wish if it were possible to treat local stations as separate entities, for the sake of distinguishing their role, similar to how AS2 Gateway does it):

PartnerListRequest listReq = new PartnerListRequest(PartnerListRequest.LIST_ALL);

/*
 you can optionally receive a filtered result based on the partner ID:
 listReq.setAdditionalListOptionStr(AS2GX);
*/

// cast() is my tiny utility method for casting the response to the appropriate type (2nd argument)

List<Partner> partners = cast(client.sendSync(listReq), PartnerListResponse.class).getList();

/*
 now you can filter the "partners" list to retrieve the interested partner and local station;
 let's call them "partnerEntity" and "stationEntity"
*/

Sending stuff out

For a send, you first have to individually upload each outbound attachment via a de.mendelson.util.clientserver.clients.datatransfer.TransferClient, accumulating the returned "hashes", and finally submit a de.mendelson.comm.as2.client.manualsend.ManualSendRequest containing the hashes along with the recipient and other information. (If you hadn't noticed, this client-based approach inherently allows you to send multiple attachments in a single message, which is not facilitated via the GUI :) )

// "files" is a String array containing paths of files for upload

// create a new file transfer client, wrapping our existing "client"
TransferClient tc = new TransferClient(client);

ManualSendRequest sendReq = new ManualSendRequest();
sendReq.setSender(stationEntity);
sendReq.setReceiver(partnerEntity);

List<String> hashes = new ArrayList<>();
List<String> fileNames = sendReq.getFilenames();

// upload each file separately
for (String file : files) {
    try (InputStream in = new FileInputStream(file)) {
        fileNames.add(Paths.get(file).getFileName().toString());
        // upload as chunks, set returned hash as payload identifier
        hashes.add(tc.uploadChunked(in));
    }
}
sendReq.setUploadHashs(hashes);

// submit actual message for sending
Throwable e = client.sendSync(sendReq).getException();
if (e != null) {
    throw e;
}
// done!

Delving into the history

Message history retrieval is fairly granular, with separate requests for list, detail and attachment queries. A de.mendelson.comm.as2.message.clientserver.MessageOverviewRequest gives you the list of messages matching some filter criteria, whose message IDs can then be used in de.mendelson.comm.as2.message.clientserver.MessageDetailRequests in order to retrieve further AS2-level details of the message.

To retrieve a list of messages:

// retrieve messages received from "sender" on local station "receiver"
MessageOverviewFilter filter = new MessageOverviewFilter();
filter.setShowPartner(sender);
filter.setShowLocalStation(receiver);
List<AS2MessageInfo> msgs = cast(c.sendSync(new MessageOverviewRequest(filter)),
    MessageOverviewResponse.class).getList();

To retrieve an individual message, just send a MessageOverviewRequest with the message ID instead of a filter:

// although it returns a list, it should theoretically contain a single message matching "as2MsgId"
AS2MessageInfo msg = cast(client.sendSync(new MessageOverviewRequest(as2MsgId)),
    MessageOverviewResponse.class).getList().get(0);

If you want the actual content (attachments) delivered in a message, just send a de.mendelson.comm.as2.message.clientserver.MessagePayloadRequest with the message ID; but ensure that you invoke loadDataFromPayloadFile() on each retrieved payload entity, before you attempt to read its content via getData().

for (AS2Payload payload : cast(client.sendSync(new MessagePayloadRequest(msg.getMessageId())),
     MessagePayloadResponse.class).getList()) {

    // WARNING: this loads the payload into memory!
    payload.loadDataFromPayloadFile();
    byte[] content = payload.getData();
}

In closing

I hope the above would help you get started in your quest for Nirvana with Mendelson AS2; cheers! And don't forget to check out our new and improved AS2Gateway, which is fully compatible with Mendelson AS2 (or any other AS2 broker, for that matter)!

3 (4) things you should know about B2B security—and about AS2, the one-stop solution

In today's digitally transformed world, security is a key concern when it comes to B2B communication—may it be a simple 210-997 exchange or a sophisticated SCM document chain. Being aware and up-to-date on B2B security will always keep you ahead of the competition, and increase your chance of becoming that "largest retailer" or "leading partner" role that you and your business has always been dreaming about.

1. What it is

B2B security naturally derives from the basics of communication security:

Confidentiality

What goes on between you and your business partner (here onwards, let's simply call him/her "partner") stays between you two; in other words, nobody else ("third party") gets to see or read whatever it is that you two are communicating. (Imagine a sealed package that can be opened only by the intended recipient.)

Eyes Only

Authenticity

When your partner receives something that seemingly came from you, he/she can verify that it indeed came from you. (Picture a signature or an official seal on a business letter.)

Authentic

Integrity

What your partner receives is exactly the same thing that you sent out from your end, and vice versa; i.e. nobody else can modify or tamper the communicated content. (Just imagine if somebody were able to change the inventory in the invoice that you just sent out, before it reaches your partner!)

Verified

Non-repudiation

Once your partner receives what you sent him/her, he/she cannot refute the fact that he/she received it. (Think about the receipt that you never forget to collect once you have made a payment.)

Agreed

2. Why you (and your partner) should care

This part should be fairly obvious, from the examples I took in the previous section: without these measures, all sorts of weird and unpleasant things could happen, eventually leading to the ruination of your business:

Hacker

  • Your rival could send your supplier a fake purchase order on your behalf—perhaps even a complete order document chain, resulting in a (surprise!) delivery at your doorstep which you never actually ordered.
  • Your rivals could intercept and view every single purchase order—or any other document, for that matter—that you would be sending to your partner.
  • Your rival could impersonate your partner, and start sending fake responses to your documents that are actually intended for your partner's eyes only. You would have no way of knowing that the other party talking is not your actual partner.
  • Even more interestingly, your rival could intercept the document you (or your partner) sent out, take it off the line, make small modifications—nothing much, maybe adding or removing a few zeroes here and there!—and put it back on the line. (I hope it's needless to explain how harmless such a "small" modification could be!)

Now that would explain why your partner would have mentioned in your negotiations, "Ah, and all documents should be exchanged securely over AS2/HTTPS" (or maybe "AS3", "OFTP" or some other weird acronym, for that matter).

3. How to achieve it

Lost your peace of mind? No worries, techniques are already in place to guard you against all of those nasty rivals out there.

WARNING: Things are about to get a little G(r)eeky. Reader discretion is advised.

Encryption for confidentiality

This is all about transforming the content that you send out, into a form that can be interpreted (read) only by your partner. (In other words, no third party would be able to make sense of the document while it is in transit.)

There are actually 2 levels of achieving this, in the "AS2/HTTPS" example above:

  • HTTPS provides transport-layer encryption, meaning whatever that is being sent on the wire (or wireless radio waves, if you insist) would be encrypted, right from the time it leaves your computer right up to the time it arrives at your partner's computer.
  • AS2 provides application-layer encryption, meaning that the application (AS2 client) triggering the sending of your document would already have encrypted it (so that it is fairly secured, even in the absence of HTTPS).

Now, if that was all Greek to you, it's just enough to keep in mind that protocols like AS2 include both, providing the security equivalent to a dual-lock safe.

Signing, for authenticity and integrity

While a real-world signature deals only with authenticity, a digital signature can also ensure integrity as well. This is due to the way a digital signature is calculated; which, in plain English, involves a kind of transformation where a concise form (a hash) of the message is created, and then scrambled (encrypted, if I may) using a special token (a private key, technically speaking) that is unique to the sending party (the technical term being digest calculation). If the receiver can unscramble the scrambled chunk using another different token (technically, the public key corresponding to the above private key), it verifies that the message had indeed been scrambled using the exact-same partnet-specific token (private key) that was mentioned earlier—and hence that it actually originated from that specific partner and nobody else.

While the unscrambled chunk of data (effectively the has that was calculated earlier by your partner, before being encrypted) can also be used to verify the integrity (intactness) of the message (e.g. you could calculate the same hash against the received message, on your end, and compare it with the hash that was sent to you by your partner), usually the MIC (message integrity code) technique is employed to explicitly enforce the integrity aspect. We'll come to that later.

By the way, if you are wondering how on earth something scrambled using one token (private key) can be unscrambled using a different token (public key), you might want to read upon asymmetric key encryption and public key infrastructure (PKI); only if you dare, of course (I'll pass).

Encryption and Signing

Receipt for non-repudiation

Just like a payment receipt, your partner who successfully received the message you just sent out, is supposed to send back a "receipt" (technically a disposition notification) saying that he/she was able to successfully receive and interpret the message. By convention, a communication is considered to have been successfully completed only after the receipt has been sent back to the sender (just like a real-world transaction). Once the receipt is sent, your partner has confirmed that he/she received whatever it is that you sent, and can never deny it.

MIC for integrity

This looks like a "repeat back to me what I just said" scheme, although it's usually more concise. Here, your partner is supposed to compute a fixed-length data chunk representing the message that he/she just received from you (a "hash" (again!), technically speaking; the chunk is usually shorter than the actual message) and send it back to you. You can then do the same computation on your end and compare your "chunk" with his/her "chunk". Since the two "chunk"s were calculated independently, if they do match, you can be pretty sure that the message (document) content is the same at both ends—and hence that your partner received exactly what you sent out.

All of the above are readily available in the AS2 (Applicability Statement 2) protocol for secure document exchange. AS2 is the most widely used B2B protocol in the modern e-commerce space, which is probably why your partner explicitly requested it. When it comes to all the nitty gritty details that we discussed above, it's all there in the official AS2 RFC, in case you are curious. (But I won't click that link, if I were you!)

However, the caveat is that AS2 requires you to have a server machine running 24×7, exposed to the public internet, so that your partners could send messages to you whenever they decide to. That in itself could be a substantial problem, especially for small- and medium-scale enterprises, with the incorporated painful set-up, configuration and testing steps, operational costs, maintenance overhead and all sorts of security concerns (remember, you are exposed to the WWW (wide, wild world) and there are hackers everywhere!).

Receipt and MIC

4. (Bonus!) How the "gateway" achieves it all—and more—for you

If you are overwhelmed about all this alien stuff by this point (or wisely skipped all the way down to here), it is perfectly logical; all those technicalities are too much for the regular, business oriented minds (and rightly so). That is precisely why we have managed AS2 clients and services—which make AS2-based trading as simple as the click of a button. Better still, now there are online AS2 solutions that free you completely from having to maintain fancy servers or other forms of hard-to-manage AS2 infrastructure.

On the cloud, light as a feather

One such solution is AS2Gateway, a cloud software-as-a-service (SaaS) that brings all those AS2 goodies—and more—right into your favourite web browser. Yup, nothing to download, install or run—just log in to your AS2G account, and you have your own dedicated AS2 space, ready for action.

AS2Gateway logo

Hit-and-run, or click-and-send

Time to forget about all those pesky security entrails that you read earlier (or wisely skipped); because AS2G does it all for you. Just log in, click the New Message icon, select your partner, pick your documents and hit Send. Under the hood, AS2G does all the heavy lifting for you—composing a message with the uploaded files, encrypting, compressing and signing it as your partner has requested, sending it to your partner, and even accepting the receipt (MDN).

24×7, all ears

When it comes to receiving documents from your partner, AS2G does an equally good job to make things super simple: it automatically accepts the incoming message, saves it in your inbox, and even sends back the receipt (MDN) based on whether the whole process was successful or not. Not to mention all the complicated stuff like message decryption, decompression, signature verification, and so on.

Messaging

Space for all, on the house!

Everything is saved under your account, securely and reliably, under stringent security and privacy standards. Just log in and go to your inbox, and everything you send and received will be right there, neatly arranged. You could view any message in detail, download its attached documents, and archive or delete old stuff to keep things tidy.

Jekyll and Hyde? No problem.

AS2Gateway allows you to manage multiple partners as well as multiple "trading stations" (meaning that you could use different identities for trading with different partners). Each partner or station can be configured totally independent of each other, giving you enormous flexibility in dealing with different business partners with varying corportate security and demands.

Ahoy, mateys! There's more!

On top of all this, AS2Gateway offers many other nifty features, including fine-grained statistics for your partners and stations, email notifications for new messages, and a free SFTP service where you can send out your documents by uploading them into your own private SFTP space and also retrieve documents from incoming messages via the same space—quite handy for integrating with your own internal systems. Coupled with a fully-fledged SFTP integrator such as the UltraESB, the end-to-end solution could soon turn out to be a game changer for your business in terms of efficiency, rapid connectivity and hands-off automation.

AS2Gateway dashboard

Looking for something in-house, on-premise?

Having read all that hands-off, cloud-hosted, zero-maintenance stuff, in case you are actually looking for an on-premise solution—one you can host in your own server, and run on your own terms—there is AS2Gateway's "big brother"—AS2Station—cut out just for the job.

AS2Gateway logo

So, the next time your partner bothers you with "secure trading", "secure B2B exchange" or "AS2", you know where to look!