Simple Integration

FUSE Ignite – First Impressions

1. What is Fuse Ignite?

In 2009, Gartner Says: Citizen Developers Will Build at Least 25 Percent of New Business Applications by 2014.

“…Future citizen-developed applications will leverage IT investments below the surface, allowing IT to focus on deeper architectural concerns, while end users focus on wiring together services into business processes and workflows,” said Eric Knipp, senior research analyst at Gartner. “Furthermore, citizen development introduces the opportunity for end users to address projects that IT has never had time to get to — a vast expanse of departmental and situational projects that have lain beneath the surface…”

That prediction is yet to come true, in my opinion, but the idea of a class of developers in the integration domain referred to as Citizen Integrators was born. Citizen Integrators are the class of integrators whose day job is not in IT Integration. These people understand the business. They are tech savvy to a certain degree but do not necessarily have a deep understanding of the underlying technology. For simple integrations, Citizen Integrators using the proper tools can do the job freeing up expensive IT Integration resources to do more complex integrations.

FUSE Ignite is the tool designed for Citizen Integrators. It is not meant to replace FUSE Standalone or FUSE on Openshift. For complex integrations, FUSE development by developers using an IDE is still the way to go.

FUSE Ignite is supported only on Openshift and comes with a FUSE subscription. Use of FUSE Online, the SaaS version of FUSE Ignite, is not part of a FUSE subscription though.

I evaluated FUSE Ignite in 2 different environments: Fuse Online and on my laptop running “oc cluster up”. My findings are documented below.

2. Red Hat FUSE Online

You can try out FUSE Online here. Just click on the Try It button and sign up for a “FREE TRIAL”.

Fuse Online
Fuse Online

2.1 Setting up Your Connections and Integration

Follow the Red Hat Fuse 7.0 Ignite Sample Integration Tutorial to set up your connections and Integrations.

2.2 Creating Integrations

Follow the manual instructions and everything will work as expected. For FUSE Developers, you can associate Connections in Ignite as Camel Endpoints; Integrations as Camel Routes and Steps as Camel Processors.

The simplest Integration I built is to set up a HTTPS connector (Consumer Camel endpoint) to a RESTful service exposed using the SaaS version of 3scale API Management Platform using the PERIODIC INVOKE URL operation. The API returns a json payload which is passed to a FTP connector ( Producer Camel endpoint) using the UPLOAD operation. This integration will invoke the HTTP service periodically and transfer the json payload using FTP to a file in a particular directory. There is no Step or Camel Processor between the endpoints.

Simple Integration
Simple Integration

Like, most FUSE developers, the first thing that comes to mind when I first saw it, is that I can do this in a few of lines of code or XML using Fuse (Camel)! As I think about it more, it dawns on me that, we are actually taking our skills in Integration too much for granted. Imagine what it takes to do this simple integration:

  • Set up an IDE eg, Developer Studio – installation is easy, learning to use the IDE may take time
  • Create a FUSE Project – not an easy task for a Citizen Integrator. Yes, there are Maven Archetypes for this but a Citizen Integrator may not know this. And when you need to add dependencies in your project, how do they know how?
  • Create the integration using either Java DSL, Blueprint or Spring Boot – you need to know Java, XML and concepts in Camel such as: Camel routes, endpoints, components, message exchange, configuring endpoints using URI, Camel context, just to name a few.
  • Test it locally – one has to know either how to run it using mvn command line or from the IDE using springboot:run.
  • Deploy to Openshift – one has to know the oc command or how to use the Openshift console. If the application requires configuration, then Openshift configmap has to be used requiring even more skill than a Citizen Integrator may have or prepared to spend the time learn.

In short, it is quite an achievement in a product that is version 1 despite some shortcomings which I am going into next.

3. Running it on “oc cluster up”

I want to install Ignite on my laptop for convenience of experimentation and demo. However, before you proceed to do this, please refer to the “Limitations” section to see if this is for you.

3.1 Installing oc Command 3.9

Download and install the oc from https://www.okd.io/download.html. It may have a newer version now. If you follow the installation instructions, you will end up using 127.0.0.1 as you server IP address which is not going to work with Ignite which requires an proper IP address, even a private one but not the loopback address.

You need to specify “public-hostname” with you server’s main IP address. For my VM, it is 10.0.2.15.

The first time you should start the oc cluster like this:

sudo oc cluster up \

--public-hostname="10.0.2.15" \

--host-data-dir='yourPathForSavingTheData' \

--host-config-dir='youPathForSavingTheConfig' \

--skip-registry-check=true

And subsequent time you start the oc cluster specifying it is to use the existing config (use-existing-config).

sudo oc cluster up \

--public-hostname="10.0.2.15" \

--host-data-dir='yourPathForSavingTheData' \

--host-config-dir='youPathForSavingTheConfig' \

--skip-registry-check=true \

--use-existing-config=true

 

3.2 Setting up FUSE Ignite

Follow the instructions here to download the installation script and install it on Openshift. The installation completes without a hitch.

3.3 Frequent Evictions

However, it takes a long time for Ignite to come up. In my case, it never did until I scaled down all other running containers. You can see a lot of eviction going on owing, I guess, to lack of resources. Ignite environment consists of 8 containers plus 2 others to provide the catalog service used as one of the endpoints for demo.

Ignite Running Containers
Ignite Running Containers

 

Evictions
Evictions

It is interesting because I’ve not set any Openshift resource limits and my VM has 2 CPUs and 10G of memory.

I even tried changing the node-config.yml as follows without any impact to the resource limitations I experienced.

kubeletArguments:

cgroup-driver:

- systemd

fail-swap-on:

- "false"

max-pods:

- "200"

pods-per-core:

- "0"

 

3.4 Limitations

Running in the “oc cluster up” environment has some challengers:

  • Frequent eviction due to lack of resources – this was highlighted in the previous section. Even after changing the node.conf, there does not seem to be any improvement. One has to resort to scaling down other applications to zero replicas to allow FUSE Ignite to run together with a simple integration. This means that you cannot use it to demo multiple integrations owing to resource limitations.
  • Unable to callback to your Ignite instance from the Internet – unless you can access your machine (physical or virtual) running “oc cluster up” from the Internet, you cannot use Salesforce or Twitter as your connection endpoints as using them requires you to register an application with them and specify a callback URL when events occur. If your main objective for running Fuse Ignite in this environment is to use it as a demo system, then you should re-examine whether not able to show Salesforce and Twitter endpoints is worth the effort to set up such an environment. Setting it up in such an environment is great for learning purposes though.

4. Conclusions

FUSE Ignite is impression as Version 1. It allows Citizen Integrators to create simple integrations. It is also good for demo’ing low code/no code integration using GUI. Customers are particularly interested in the Data mapper.

In my experience, the sentiment of the customers after seeing my Ignite demo is usually quite extreme. Some immediately say: “I can use it here…I can use it there…”. While others say: “My integrations are much more complex than that. We won’t be using it”.

In my opinion, to make Ignite widely accepted by the new class of Citizen Integrators, we need to fix a number of issues and enhance the product. Improvement areas include:

  • Providing more information when integration is not working. How does one debug an integration?
  • Adding more Connectors and Steps
  • Allowing import of Ignite integration to Developer Studio for further enhancement and deployment
  • Supporting multi-tenancy instead of having to run up to 9 containers for each user’s Ignite instance

FUSE Ignite, with improvements, will fulfill the needs of a class of users we call Citizen Integrators. And if Gartner prediction is accurate, iPaaS is the fastest growing segment of the Integration Market by 2021. With an improved FUSE Ignite, we shall have that market covered.