Wednesday, March 15, 2017

UltraStudio in Action, Episode 1: Write Your Own Basic Authenticator!

Basic authentication is perhaps the easiest—though not the most secure—way to control access to your in-house APIs. It allows users to gain access to the API simply by providing their username-password credentials, without the need of advanced encryption or third-party involvement as in OAuth.

We can easily try out basic authentication using UltraESB-X, an easy-to-use ESB (enterprise service bus) with a graphical UI for developing and testing your project on-the-fly. Utilizing the flexibility of the Project-X framework powering UltraESB-X, you can easily write a simple custom processing element to get hands-on experience with a simple basic authentication flow.

For our scenario we shall use the UltraStudio IntelliJ IDEA plugin, which comes bundled with an UltraESB-X instance and provides the graphical UI for developing our little project. UltraStudio can be obtained from here. The license key, which you will receive in a confirmation email, is good for 30 days with the embedded UltraESB-X, i.e. it provides you with a free 30-day trial period for playing around with the product. (Of course you can continue to use UltraStudio even after that as well, although the embedded ESB will require a new license or an extension; or you can build an independent project archive and run it in an external UltraESB-X or in IPS.)

Having said that, let's get to work!

Getting UltraStudio up and running

  1. Download IntelliJ IDEA (if you don't have it already). You may want to upgrade to at least 2016.1 if what you have is older. The Community Edition is free for unlimited use, so no issue there!
  2. Download the UltraStudio plugin for your platform: Linux, Windows or Mac.
  3. Follow the installation guide to install the downloaded plugin.

Now you're ready for all your adventures with UltraStudio!

Getting an API up and running

UltraStudio has a set of pre-built sample projects. One of them, REST service mediation, can be used as the base REST API for our basic auth example. The 'API' doesn't do much by itself—it just acts as a proxy, forwarding incoming queries to openweathermap.org—but it gives us a good starting point for trying out basic auth.

  1. First, make sure you have internet connectivity as UltraStudio pulls some of its resources from online.
  2. In IDEA, select File → New → Project.
  3. In the New Project dialog, select Sample Ultra Project on the left sidebar, and click Next.
  4. Ultra Projects (collections of integration flows, or the successor of deployment units in case you are familiar with the original UltraESB) are Maven projects. So let's enter a Maven Group ID and Version for our project, and also a Base Package name sample.auth.basic.
  5. On the next window, you will see all samples available for UltraStudio. Point to REST Service Mediation and click Download.
  6. When the download is complete, UltraStudio will display a confirmation "Successfully Completed the Download. Click Next to Continue"; at this point, click Next.
  7. In the next step, select the location for saving the new project.
  8. Finally, click Finish to create the project.

The project needs no additional configurations and can be run right away!

  1. Click Reimport All Maven Projects button on the Maven Projects tool window, for IDEA to detect the newly created project as a Maven project.
  2. Click Run → Edit Configurations.
  3. Click + button on the Run/Debug Configurations dialog and add a new UltraESB Server run configuration.
  4. Give a name to your configuration (e.g. RESTProxy) and click OK.
  5. Click Run → Run RESTProxy to run your project in the embedded ESB!

To ensure things are working fine,

  1. Check the last few log lines in the Run window. They should indicate something like:
    2017-03-12T20:41:34,213 [127.0.1.1-janaka-ENVY] [main] [RestServiceMediation-17.01] [310007I010]  INFO NIOHttpListener HTTP NIO Listener on port : 8280 started
2017-03-12T20:41:34,217 [127.0.1.1-janaka-ENVY] [main] [system-] [145001I013]  INFO XContainer AdroitLogic UltraStudio UltraESB-X server started successfully in 5 seconds and 976 milliseconds
2017-03-12T20:41:34,219 [127.0.1.1-janaka-ENVY] [HttpNIOListener-8280] [system-] [310007I012]  INFO NIOHttpListener IO Reactor started for HTTP NIO Listener on port : 8280 ...
  2. Follow the instructions mentioned in the sample to send a request to the API, and verify that a valid response is received.

Plugging in Basic Auth

Now that we have a working API (so to speak) we can think of how to 'secure' it with basic auth:

  • Anonymous users should be blocked with an "unauthorized" message.
  • Users that provide an invalid (malformed) authentication header should be blocked with an appropriate error message.
  • Users that provide an invalid username/password should be blocked with just the same error message.
  • Only successfully authenticated users should be allowed to move forth with the API call.

Accordingly we can easily fulfill our requirement by including a custom Basic Authenticator processing element on the flow. Since credentials are simple username-password pairs, we shall load them from a CSV file at startup.

  1. To create a custom processing element, expand src/main/java in the Project tool window, right click sample.auth.basic package and select New → Processing Element.
  2. Provide BasicAuthenticator as the class name.
  3. Modify the content of the newly created source file, as follows:
    • Change the extends clause to AbstractSequencedProcessingElement.
      public class BasicAuthenticator extends AbstractSequencedProcessingElement {
    • Enter value "Basic Authenticator" for attribute displayName of the @ProcessingElement class-level annotation. This is the name that would be displayed in the flow editor.
      @Processor(displayName = "Basic Authenticator", type = ProcessorType.CUSTOM)
    • Add a Map instance variable credentialCache to hold the username-password credentials for authentication.
          private final Map credentialCache = new HashMap<>();
    • The process() method is where the magic happens, where we evaluate the user request for valid credentials. For this we,
      1. extract the Authorization header from the HTTP request,
                XMessage msg = xMessageContext.getMessage();
        Optional authHeader = msg.getFirstStringTransportHeader("Authorization");
      2. decode its credential portion,
                    String authString = new String(Base64.getDecoder().decode(authHeader.get().substring(6)));
            String[] credentials = authString.split(":", 2);
      3. and see if if matches any of the entries in our credentialCache.
                    if (!credentials[1].equals(credentialCache.get(credentials[0]))) {
      4. If the Authorization header is either absent or malformed (i.e. not a base 64-encoded string made of a colon-separated username-password pair), we modify the message to a 401 HTTP response with an appropriate message and raise an exception to trigger a response with it via the error path.
                if (!authHeader.isPresent()) {
            throw deauthorize(msg, "Please provide an Authorization header for basic auth");
        }
...
            if (credentials.length != 2) {
                throw deauthorize(msg, "Authorization header is not in the correct format");
            }
            if (!credentials[1].equals(credentialCache.get(credentials[0]))) {
                throw deauthorize(msg, "Invalid username/password");
            }
...
        } catch (Exception e) {
            throw deauthorize(msg, "Failed to process the Authorization header: " + e.getMessage());
        }
      5. If everything tallies out, we allow the message to continue by returning an ExecutionResult.SUCCESS from the element.
                    msg.removeTransportHeader("Authorization");
            return ExecutionResult.SUCCESS;
      6. We need to ensure that our credential map credentialCache is populated by the time our API proxy endpoint is up and serving. We can achieve this using the initElement() method of the processing element, which gets called when the flow (and hence the element) is being initialized at server startup. At the moment we shall simply read a credentials.csv file inside the src/test/resources directory and load its content to our credential map, treating it as a two-column list of username-password pairs:
                try (InputStream is = new FileInputStream(getResource("credentials.csv").getPath())) {
            Scanner scanner = new Scanner(is);
            while (scanner.hasNextLine()) {
                String[] values = scanner.nextLine().split(",");
                credentialCache.put(values[0], values[1]);
            }
        } catch (IOException e) {
            throw new IntegrationRuntimeException("Failed to initialize credential cache", e);
        }

The completed class should look like:

package sample.auth.basic;

import org.adroitlogic.x.annotation.config.Processor;
import org.adroitlogic.x.api.ExecutionResult;
import org.adroitlogic.x.api.IntegrationRuntimeException;
import org.adroitlogic.x.api.XMessage;
import org.adroitlogic.x.api.XMessageContext;
import org.adroitlogic.x.api.config.ProcessorType;
import org.adroitlogic.x.base.format.StringFormat;
import org.adroitlogic.x.base.processor.AbstractSequencedProcessingElement;
import org.springframework.context.ApplicationContext;

import java.io.FileInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.util.*;

@Processor(displayName = "Basic Authenticator", type = ProcessorType.CUSTOM)
public class BasicAuthenticator extends AbstractSequencedProcessingElement {

    private final Map credentialCache = new HashMap<>();

    @Override
    protected ExecutionResult sequencedProcess(XMessageContext xMessageContext) {
        XMessage msg = xMessageContext.getMessage();
        Optional authHeader = msg.getFirstStringTransportHeader("Authorization");
        if (!authHeader.isPresent()) {
            throw deauthorize(msg, "Please provide an Authorization header for basic auth");
        }

        try {
            String authString = new String(Base64.getDecoder().decode(authHeader.get().substring(6)));
            String[] credentials = authString.split(":", 2);
            if (credentials.length != 2) {
                throw deauthorize(msg, "Authorization header is not in the correct format");
            }
            if (!credentials[1].equals(credentialCache.get(credentials[0]))) {
                throw deauthorize(msg, "Invalid username/password");
            }
            msg.removeTransportHeader("Authorization");
            return ExecutionResult.SUCCESS;

        } catch (IllegalStateException e) {
            throw e;
        } catch (Exception e) {
            throw deauthorize(msg, "Failed to process the Authorization header: " + e.getMessage());
        }
    }

    private IllegalStateException deauthorize(XMessage msg, String reason) {
        msg.setResponseCode(401);
        msg.setPayload(new StringFormat(reason));
        return new IllegalStateException("Authentication failed for message " + msg.getMessageId());
    }

    protected void initElement(ApplicationContext context) {
        try (InputStream is = new FileInputStream(getResource("credentials.csv").getPath())) {
            Scanner scanner = new Scanner(is);
            while (scanner.hasNextLine()) {
                String[] values = scanner.nextLine().split(",");
                credentialCache.put(values[0], values[1]);
            }
        } catch (IOException e) {
            throw new IntegrationRuntimeException("Failed to initialize credential cache", e);
        }
    }
}

Now let's modify the integration flow to include our basic auth element:

  1. While having the BasicAuthenticator loaded in the editor, click Build > Recompile BasicAuthenticator.java to compile the class.
  2. Now open the file src/main/conf/simple-rest-service-mediation.xcml (and switch to the Design tab at the bottom). If the file is already open, click the Refresh View button inside the UI to re-render the view.
  3. Once the refresh is done, you will find the new Basic Authenticator element under Processors → Custom category on the left palette. Drag-and-drop it into the work area.
  4. Rewire the flow such that, before hitting the HTTP Egress Connector, the message flows through the Basic Authenticator:
    1. Delete the path going out from NIO HTTP Listener's Processor port.
    2. Connect the above port to the Input port of Basic Authenticator.
    3. Connect the Next port of Basic Authenticator to the Input port of NIO HTTP Sender.
    4. Connect the On Exception port of Basic Authenticator to the Input (bottommost) port of NIO HTTP Listener (so that it now has 2 incoming connection paths).

Now run the previously created RESTProxy configuration to get the ESB running.

Testing the whole thing

To verify that our basic authenticator indeed works,

  1. Try to access the /service/rest-proxy endpoint in the same way you tried in the original sample (the configuration should still be available in the Ultra Studio Toolbox unless you had already closed IDEA). It would fail with an error response similar to the following, as we are not sending an Authorization header:
    HTTP/1.1 401 Unauthorized
User-Agent: AdroitLogic (http://adroitlogic.org) - SOA Toolbox/1.5.0
Host: localhost:8280
Date: Mon, 13 Mar 2017 01:13:53 GMT
Server: AdroitLogic UltraStudio UltraESB-X
Content-Length: 53
Content-Type: text/plain; charset=ISO-8859-1
Connection: close

Please provide an Authorization header for basic auth
  2. Now let's try an invalid Authorization header.
    1. Switch to the configuration view on the Toolbox HTTP client (via the Show/Hide config panel button).
    2. Switch to the Custom Headers tab.
    3. Add a custom header with key Authorization and a malformed value base64 encoded values cannot contain spaces.
    4. Now, sending another request, we see another 401 response indicating tha our malformed Authorization header could not be decoded:
      HTTP/1.1 401 Unauthorized
Authorization: base64 encoded values cannot contain spaces
User-Agent: AdroitLogic (http://adroitlogic.org) - SOA Toolbox/1.5.0
...

Failed to process the Authorization header: Illegal base64 character 20
  3. Now, for an invalid username/password:
    1. Remove the previous custom header (using the - button).
    2. Switch to Authentication tab.
    3. Enter a username that is not included in the credentials CSV (or a correct username with a wrong password; both would have the same effect) and try again:
      HTTP/1.1 401 Unauthorized
Authorization: Basic YWRtaW46cGFzc3dvcmQ=
User-Agent: AdroitLogic (http://adroitlogic.org) - SOA Toolbox/1.5.0
...

Invalid username/password
      See? It's working! But we need to verify one more thing: whether a correct username-password pair can actually get us through :)
  4. On the Authorization tab, enter a correct username-password pair and try again. You shall receive a 200 response with the intended payload.
    HTTP/1.1 200 OK
Access-Control-Allow-Origin: *
Access-Control-Allow-Credentials: true
Access-Control-Allow-Methods: GET, POST
X-Cache-Key: /data/2.5/weather?APPID=________________________________&q=london
Date: Mon, 13 Mar 2017 01:25:32 GMT
Content-Type: application/json; charset=utf-8
Server: AdroitLogic UltraStudio UltraESB-X
Content-Length: 449
Connection: close

{"coord":{"lon":-0.13,"lat":51.51},"weather":[{"id":803,"main":"Clouds","description":"broken clouds","icon":"04n"}],"base":"stations","main":{"temp":281.94,"pressure":1022,"humidity":81,"temp_min":280.15,"temp_max":284.15},"visibility":10000,"wind":{"speed":3.1,"deg":300},"clouds":{"all":75},"dt":1489366200,"sys":{"type":1,"id":5091,"message":0.0061,"country":"GB","sunrise":1489385925,"sunset":1489428120},"id":2643743,"name":"London","cod":200}
  5. Finally, let's also see what would happen if we are successfully authenticated but the backend is unavailable:
    1. Disconnect from the internet.
    2. Retry the last request.
    3. You shall receive a 500 error, while a corresponding exception (possibly UnknownHostException) would be thrown on the ESB log:
      HTTP/1.1 500 Internal Server Error
Date: Mon, 13 Mar 2017 01:27:29 GMT
Server: AdroitLogic UltraStudio UltraESB-X
Content-Length: 21
Content-Type: text/plain; charset=ISO-8859-1
Connection: close

Internal Server Error
      ESB log:
      2017-03-13T06:57:29,908 [127.0.1.1-janaka-ENVY] [pool-2-thread-8] [system-] [110801E005] ERROR MessageReceiver Error occurred while processing message context : ae96376f-4ba0-0539-0000-000000000005
 java.net.UnknownHostException: api.openweathermap.org
	at org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.validateAddress(DefaultConnectingIOReactor.java:245) ~[httpcore-nio-4.4.jar:4.4]
	at org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.processSessionRequests(DefaultConnectingIOReactor.java:264) ~[httpcore-nio-4.4.jar:4.4]
	at org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.processEvents(DefaultConnectingIOReactor.java:141) ~[httpcore-nio-4.4.jar:4.4]
	at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor.execute(AbstractMultiworkerIOReactor.java:350) ~[httpcore-nio-4.4.jar:4.4]
	at org.adroitlogic.x.transport.http.nio.AbstractNIOHttpSender.startIOReactor(AbstractNIOHttpSender.java:195) ~[x-transport-nio-http-17.01.jar:?]
	at org.adroitlogic.x.transport.http.nio.AbstractNIOHttpSender.access$500(AbstractNIOHttpSender.java:65) ~[x-transport-nio-http-17.01.jar:?]
	at org.adroitlogic.x.transport.http.nio.AbstractNIOHttpSender$3.run(AbstractNIOHttpSender.java:174) ~[x-transport-nio-http-17.01.jar:?]
	at java.lang.Thread.run(Thread.java:745) [?:1.8.0_65]
2017-03-13T06:57:29,915 [127.0.1.1-janaka-ENVY] [pool-2-thread-8] [system-] [310401E006] ERROR XHttpAsyncRequestHandler Error occurred while processing the message java.net.UnknownHostException: api.openweathermap.org

While this is only a very crude example of basic auth, the flexibility of the new processing element model of UltraESB-X permits you to go to whatever level of sophistication required for robust operation, yet maintaining a clean and easy-to-maintain integration flow.

at

No comments:

Post a Comment

Thanks for commenting! But if you sound spammy, I will hunt you down, kill you, and dance on your grave!