Onesait Platform Release 2.3.0-immortal

Once again, this quarter we bring you here the new release of the Onesait Platform, reaching its version 2.3, which you have helped us to name as «Immortal», following as you know the version policy that we have established.

With thism we complete the roadmap for 2020, having reached almost all the points that we had decide.

So, let’s see what new features we have included in this Q4.

---

Engine

This version supports the development of microservices and all types of applications. To simplify the development of these functionalities, the platform allows for visual development with a LowCode approach, including a centralised web console that supports administration, configuration, development and deployment.

In this version we have incorporated important features:

Centralized Log

This is a functionality that you had requested several times, and although the CaaS platforms already support this, we have decided to incorporate a centralised log management solution.

We have decided to base it on Graylog, an open-source solution specialised in the centralised management of logs. This also allows us to take advantage of previous work that other colleagues did in the Onesait Revolution initiative (specifically the onesaitplatform-revolution-onelog team).

In this first version of the functionality, the platform’s components’ logs have been centralised, and we explain how to adapt the logs of other components (for example, a microservice) allowing for complete searches to be performed by component, time, log type, etc.

In the next release, we intend to integrate Graylog’s UI into the Platform’s Control Panel.

You can find more information about this functionality in the following article: Centralised logs.

Integration of Keycloak as Advanced Identity Manager

As you know, the Platform includes an Identity Manager built on top of Spring Cloud which covers typical security use cases such as authentication, authorisation, realms, plugins, etc. (have a look at the Identity Manager guides if you want to know more about this).

We have detected that in some cases, this functionality is not enough in the Projects and Products; for example, when you need to authenticate against multiple user repositories and authentication providers (user federation), integration with other IMs (identity brokering), personalisation of the authentication flow and sessions, etc. This is why we have decided to incorporate Keycloak as an alternative Identity Manager for these scenarios, as it is a reference Open Source solution in this field.

In this integration, we have mapped Keycloak’s Realms with the Platform verticals, the Keycloak Clients with the Platform’s Realms; and we use the Platform ConfigDB as the default Provider (although LDAP can also be used).

You can find more information about the integration in this guide: Support for Keycloak as an Advanced Identity Manager.

Generation of web applications Vue.js from FIGMA design

Nowadays, FIGMA has become the standard tool for designing the UIs of application – it is, for example, the tool used in Onesait UX equipment.

From the figma-low-code plugin created by Klaus Schaefers (who is a real reference in the Open Source world in this field), and from the information on the variables and methods introduced in the plugin within the Platform, it is possible to map that information with methods from our REST APIs.

From there, the Platform is capable of generating a fully operative Vue.js application, in which we can also integrate the Platform’s Dashboards in a very simple way.

You can find a user guide here: Generating Vue.js web applications from a FIGMA design.

(New) centralized configuration

Although this functionality already existed, it was intended mostly for by Platform administrators, and for the Platform internal use. In this release, the functionality has been remodelled so that developers can create their own configurations and also use them from their own microservices or applications.

Interested? You can find more information in the section: Central Configuration Guides.

Query export from Query Tool

Many of you were missing a utility that would allow you to export the data of a query made from the Query Tool in several formats. At last, here it is!

The utility allows you to export the query results in CSV and JSON formats. Besides, if you are an administrator user, the limit of records configured by the Platform will not be applied, so you will be able to export all the data in a query.

This is explained in the Query Tool Guides.

Cache in APIs

When defining an API on the Platform, a cache interval (in minutes) can be entered as a configuration attribute. This will be the time during which the records will be valid in the cache. Once this period of time has passed, the information on the data origin will be recovered again.

You can see how this functionality works in detail in the following guide: Cache in APIs.

Asynchronous launch of import/export tool

In this version, we have improved the tool that allows you to export and import the elements of the Platform so that the launch is asynchronous and so that you don’t have those uncomfortable timeouts that are always bothering so much.

You can find more information in the Export-Import Tool Guides section.

Various improvements

As you know, we are always mindful of your opinions and comments, so in this release we have incorporated several of the improvements that come from your requests – both those made via support tickets and those made from Delibera, a new tool that we are using and that has allowed us to participate more globally by applying Collaborative Intelligence.

The applied improvements have been:

• Paging in the list of Realms users:
  

• Configuration to not load sample data in the Platform’s installation.
• Bulk support of upserts in the UPDATE component of Dataflow:
  

• Improvements in APIs of Management of the Open Data component.
• Improvements in the new SQL Engine: supporting operators (pj select 2+3*6/(5-3) from Restaurants, unions, etc.
• Physical deletion of users (from list and detail).
  

---

Intelligence

This distribution of the Platform supports the development of systems that make use of Intelligence capabilities, either with its AI capabilities, or ingesting from different sources, analysis, visualization, etc.

During this quarter, we have worked on:

DocumentDB support in DataHub

From this release, you can use DocumentDB as a possible persistence engine supported by the Platform (you know we support any relational database, MongoDB, Elasticsearch, Kudu, HIVE, CosmosDB, etc.).

DocumentDB is an agile, scalable, high-availability and fully-managed document database service on Amazon AWS that is fully compatible with the MongoDB API and operation.

Thanks to the new SQL query engine, we can use one instance of DocumentDB as if it were a MongoDB 4.0, allowing that all the development done be compatible.

In the following link you can see the Storage Engines that are supported in the Platform: Storage Engines.

AuroraDB support in Semantic DataHub

Amazon Aurora is a MySQL- and PostgreSQL-compatible relational database offered by AWS as a service.

The integration with AuroraDB allows us to use it as ConfigDB for Platform deployments, but also as a relational database on which we can manage platform concepts.

REST API for DataRefiner component

In this release, we have extended the APIs with one to be able to access the DataRefiner component via REST API, thus automating the loading and cleaning of data coming in various formats (such as XSL, CSV, XML, JSON) before loading it on the Platform.

---

Things

This distribution of the Platform supports the development of IoT systems, both in the Cloud and Edge environments.

In this quarter we have made progress in:

Synoptic engine extensibility with external SVG libraries

Our powerful synoptic engine is based on SVG for the creation of the entire SCADA-style UI. In this release we have incorporated an improvement that allows us to load new SVG component libraries to compose our synoptics:

---

If you are interested in knowing more about it, don’t forget to visit the guide about the possibility of including image libraries in synoptics.

DevOps

Within this section we include all the tools, utilities and platform capabilities that help in the Development and Operation.

New Operations Manual

In this version we have reviewed and improved all the existing documentation on the operation of the Platform to create a complete operations manual.

This manual is intended to serve as a basis for the teams that are using the Enterprise version and need to know the operation of the Onesait Platform Engine & Intelligence. It includes basic operation, special operations and troubleshooting.

Inclusion of Billing in Operation Console

We keep on working on the Operation Console, and have added the Billing functionality.

From this section, the user will be able to consult the monthly cost of each of his environments in each Project, as well as the total monthly cost of the whole Project, including in this cost the Platform subscriptions (type, size, modules) and the size of the OpenShift namespace:

Platform distributions on Helm Charts

In this release, we have added important improvements to the Kubernetes Cluster Platform deployment Chart Helm, as Openshift.

Starting from the different distributions of the Platform, the Chart has been divided into independent Charts following the same logic. Besidesn, work has been done to further automate the deployment process, so that virtually no manual intervention is required.

You can find more information in the following article: Platform distributions in Helm Charts.

Signed distribution of platform modules with Docker Content Trust

From this version onwards, when generating the Platform’s Docker images, these are signed to guarantee that the distributed images have not been altered in the process, something known in Docker as DCT (Docker Content Trust):

You can find all the information about this in the following guide: Signed distribution of Platform modules with Docker Content Trust.

---

Onesait Platform Community

This line of work includes all the tasks related to the Platform Community of which the Onesait Platform Open Source version is a part, as well as the different communication channels of the Platform.

With regard to the strategy established for 2021 on the lines of the Community, during this last quarter we have worked on the following points:

Onesait Platform Webcast

Last Thursday, December 17th 2020, we carried out our first webcast about the Platform, where we explained how to install this Community version of the Onesait Platform in two different environments; one on Linux and the other on Windows (video only available in Spanish for now).

We told you about this not long ago in the blog: Onesait Platform Webcast.

Community Channels

We continue to increase our presence in web channels and social networks around our community:

• We have published several articles in DZone, with more than 43,400 readings! A success for a first time.
• 45 new users in the Onesait Platform CloudLab environment. There are already more than 900 users using our platform in the Cloud.
• We have grown to 240 developers in Github. In this period we have made more than 50 clones of our Community version of the Platform.
• We have increased our presence on Twitter, with more than 200 published tweets and 44,600 total impressions.
• We are growing in content on the Blog, both in Spanish and English, with 38 new entries.
• New subscribers to our YouTube channel, adding 45 hours of viewing time and more than 6,249 impressions.

✍🏻 Author(s)