Showing posts with label drag-n-drop integration. Show all posts
Showing posts with label drag-n-drop integration. Show all posts

Sunday, November 19, 2017

Stop pulling out your (JSON-ey) hair; just drag, drop and connect!

The app is finally taking shape.

Data is sitting in your datastore.

Users are about to start bombarding the front-end with requests.

Quite a familiar scenario for any standard web/mobile app developer.

You have approached the Big Question:

How to get the balls rolling?

How to transform user actions into actual backend datastore operations?

One (obvious) way would be to build an ORM, configure a persistence provider (such as Hibernate-JPA) and link the pieces together through an MVC-style contraption.

But what if you don't want all those bells and whistles?

Or, what if all you need is just a quick 'n dirty PoC to impress your client/team/boss, while you are struggling to get the real thing rolling?

Either way, what you need is the "glue" between the frontend and the data model; or "integration" in more techy jargon.

UltraESB-X, successor of the record-breaking UltraESB, is an ideal candidate for both your requirements. Being a standalone yet lean runtime—just 9 MB in size, and runnable with well below 100 MB of heap—you could easily deploy one in your own dev machine, prod server, cloud VM, Docker, or on IPS, the dedicated lifecycle manager for on-premise and (coming up) cloud deployments.

UltraESB-X logo

As if that wasn't enough, building your backend becomes a simple drag-and-drop game, with the cool UltraStudio IDE for integration project development. Just pick the pieces, wire them together under a set of integration flows—one per each of your workflows, with interleaving subflows where necessary—and have your entire backend tested, verified and ready for deployment within minutes.

UltraStudio logo

We have internally used UltraESB-X seamlessly with JPA/Hibernate, whose details we hope to publish soon—in fact, there's nothing much to publish, as it all just works out of the box, thanks to the Spring-driven Project-X engine powering the beast.

Project-X logo

That being said, all you need right now is that QnD solution to wow your boss, right?

That's where the JSON Data Service utility comes into play.

JSON data service

Tiny as it may seem, the JSON Data Service is a powerful REST-to-CRUD mapper. It simply maps incoming REST API requests into SQL, executing them against a configured database and returning the results as JSON. Exactly what you need for a quick PoC or demo of your app!

We have a simple yet detailed sample demonstrating how to use the mapper, but all in all it's just a matter of specifying a set of path-to-query mappings. The queries can utilize HTTP path and query parameters to obtain inputs. SQL column name aliases can be used to control what fields would be returned in the response. The HTTP method of the inbound request (GET, POST, PUT, DELETE) decides what type of operation (create, read, update, delete) would be invoked. Of course, you can achieve further customization (adding/modifying fields, transforming the result to a different format such as XML, as well as audit actions such as logging the request) by simply enhancing the integration flow before the response is returned to the caller.

For example, here are some REST API operations, with their corresponding JSON Data Service configurations (all of which could be merged into a single integration flow, to share aspects like authentication and rate limiting):

Assuming

  • a book API entity to be returned to the frontend:
    {
    	"name": "book_name",
    	"author": "author_name",
    	"category": "category_name"
    }
  • a BOOK table:
    (
    	ID SMALLINT,
    	NAME VARCHAR(25),
    	AUTHOR_ID SMALLINT,
    	CATEGORY VARCHAR(25)
    )
  • and an associated AUTHOR table:
    (
    	ID SMALLINT,
    	NAME VARCHAR(25)
    )

The following API endpoints:

REST path operation
GET /books?start={offset}&limit={count} return all books, with pagination (not including author details)
GET /books/{id} return a specific book by ID, with author details
GET /books/search?author={author} return all books of a given author

could be set up with just the following data service configuration mapping (the rest of the steps being identical to those in our dedicated sample; just ensure that you maintain the order, and note the extra SINGLE: in front of the 2nd query):

JSON Data Service: Simplest Flow

key value
/books/search?author={author:VARCHAR}
SELECT B.NAME AS name, B.CATEGORY AS category
    FROM BOOK B, AUTHOR A
    WHERE B.AUTHOR_ID = A.ID AND A.NAME = :author
/books/{id:INTEGER}
SINGLE: SELECT B.NAME AS name, A.NAME AS author, B.CATEGORY AS category
    FROM BOOK B, AUTHOR A
    WHERE B.AUTHOR_ID = A.ID AND B.ID = :id
/books?start={offset:INTEGER}&limit={count:INTEGER}
SELECT NAME AS name, CATEGORY AS category 
    FROM BOOK LIMIT :offset, :count

See? Anybody with a basic SQL knowledge can now set up a fairly complex REST API, without writing a single line of code, thanks to the JSON Data Service!

Project-X S01E01: Pilot

P.S.: Okay, while the Pilot is supposed to arouse curiosity and keep you on the edge of the seat for S01E02, I'm not sure how well I've done that—rereading what I just completed writing. Anyway, see for yourself!

In February 2017, something happened. Something that has never been seen, and rarely been heard, ever before.

Project-X.

Project-X movie poster

No, not the movie! That was in 2012!

Engine. Messaging Engine.

Project-X logo

A lean (~9 MB), fast (yup, benchmarked) and reliable messaging engine.

But not just a messaging engine.

The original authors supposed it to be an Enterprise Service Bus (ESB). But, over time, it evolved.

Into a simple yet formidable integration middleware product.

Capable of connecting the dots: you, your customers, your partners, and the wide, wide world (WWW).

to the rescue!

Proven by powering the cores of many other integration solutions, including the famous B2B AS2 trading platform.

And, most importantly, something that can help you—and your company—tackle its integration challenges at a fraction of the time, effort and cost. Something that would allow you to draw your solution rather than coding or configuring it. Something that would make your PoC as good as your final draft, as they would essentially be the same.

Something that would make you enjoy integration, rather than hating it.

While you can always explore our official documentation to unravel how all this works (yup, I heard that yawn :)) the following couple of facts is all you need to know in order to grab Project-X by its horns:

Project-X deals with messages, or distinct and quantifiable bits of information (a clock tick, a HTTP request, an event/message over JMS/Kafka, a file dropped into your SFTP folder, a neutron emitted from a U-235, an interplanetary collision,... you name it).

U-235 fission

The beauty is the fact that almost every bit of interaction in an integration (connecting-the-dots) scenario can be mapped into a message. Care to see some examples?

  • "A consumer is calling your API" maps into "a HTTP request"
  • "Your customer just bought 3 techno-thrillers from your ebook store" could map to "an event over Kafka"
  • "Your partner just sent you an invoice" could map into "a file dropped into your SFTP folder"

Being the expert on messages, Project-X can take up the rest.

Project-X magical expertise

Project-X can consume messages from an insanely wide variety of sources. Ingress connectors bring in these messages into the Project-X engine. We already have connectors for all of the above examples, except maybe for the last (left as an exercise for the reader).

NIO HTTP ingress connector

Project-X can emit messages into an equally wide array of destinations over different media. Egress connectors are the ones that do this.

NIO HTTP egress connector

In between consumption and emission, a message can go through all sorts of weird experiences, which could consist of transormation, extraction, enrichment, conditional branching, looping, split-aggregation, throttling, exceptions,... and a lot more of other unimaginable stuff. Processing elements perform all this magic.

header conditions evaluator

(By the way, This document deals with all the gruesome details, in case you are interested.)

An ingress connector, a chain of processing elements and an egress connector, together make up an integration flow that represents everything that a message is destined to go through, given that it is sucked in by the ingress connector. A simple analogy is a conveyor belt on a production line, where something comes in and something (similar or different) comes out. Forget not, however, that depending on your requirement, an integration flow can be made as complex, rich and flexible as you like; your imagination being the only limit.

integration flow using the above components

Related integration flows are often bundled into a single integration project (yup, we love that word "integration"), which can be deployed as a single unit of functionality in a Project-X engine such as UltraESB-X.

integration architecture

Project-X keeps track of each and every message flying through the integration flows in every single project that is deployed in its runtime, and fully manages the lifecycle of each message (including allocating its required resources, routing it to the correct connector, pumping it through the flow, handling failures, gathering metrics, and cleaning up after its death/completion).

On top of all this, Project-X has its own bag of tricks, goodies and other cool stuff:

For you, the tech-savvy:

zero-copy proxying

For you, developers and architects:

  • A five-minute prototyping platform for your next integration project, whose thin-slice PoC outcome would eventually evolve into a fully-blown production deployment
  • An intuitive, DIY integration surface with the familiar notions of messages, events, connectors and flows, instead of painful XML configurations
  • A warehouse of ready-made connectors and processing elements to choose from
  • A simple and flexible way to create your own connectors and processors, and share/reuse them across projects (so you never have to reinvent the wheel across your colleagues, teams or departments... or even companies)
  • A super-cool IDE where you can graphically compose your integration solution (drag-and-drop, set-up and wire-together) and test, trace and debug it right away

UltraStudio integration flow message tracing

For you, deployers and sysadmins:

  • A pure Java-based runtime (no native libs, OSGi, voodoo or animal sacrifices)
  • Pluggable, REST-based, secure management API for remote administration
  • Pluggable analytics via Elasticsearch metrics collectors
  • Ready-made runtime bundled with the IDE for seamless dev-test cycles
  • Quick-and-dirty testing with ready-made Docker images: choose between slim and rich
  • A production bundle complete with a daemon (service-compatible) deployment, and in-built statistics and management servers
  • A tooling distribution for management operations: a CLI for engine governance, and ZooKeeper client/server bundles for clustering
  • A fully-fledged management console for tracking, monitoring and managing your deployment, complete with statistics, alerting and role-based access control
  • A fully-fledged containerized deployment platform, that can run in-cloud or on-premise, for deploying and managing Project-X clusters with just a few mouse clicks (and keystrokes)

Project-X integration solution development cycle

To be continued...

(Keep in touch for S01E02!)