Data Gateways Basics

Version 1.0

Last updated: April 22, 2020

1. What is Data Gateways?

Data Gateways is an easy-to-use, centralized method of exporting data out of Blue Prism for use in external systems.

Blue Prism generates a variety of data as it is used which can be useful in other systems outside of Blue Prism. An easy way to get that data out of Blue Prism is by utilizing Data Gateways.

1.1 What Kinds of Data Can Be Exported?

  • Published Dashboards

These are dashboards created in Blue Prism that monitor the performance and status of the Blue Prism environment.

  • Session Logs

These are detailed logs of every step a Digital Worker takes when executing a process.

  • Custom Object Data

There is a Data Gateways Visual Business Object (VBO) that can be deployed in a Blue Prism Process. The VBO can gather specific, customized pieces of data to export via Data Gateways. This is useful if there is a requirement to export data that would not otherwise be captured in a Session Log or Published Dashboard.

1.2 Where Can Data Be Exported?

There are four destinations to which Data Gateways can export data.

  • File
  • HTTP Endpoint
  • Splunk
  • Database

2. Why Use Data Gateways?

Of the many use cases for Data Gateways, three are highlighted below:

  • Disaster Recovery:

There may be a requirement for an organization to maintain backup copies of Session Logs outside of the Blue Prism database as part of a disaster recovery plan. Data Gateways can automatically send Session Logs to a file server for long-term, backup storage.

  • Data Analysis:

Data Gateways can export Session Logs to a Splunk instance which can then be used to identify performance trends over time. This would allow a Blue Prism user to identify and target which Processes are most error-prone.

  • Data Visualization:

Data Gateways can export Published Dashboards so they can be incorporated into other data visualization tools. Perhaps an IT organization has a requirement to maintain a single dashboard that provides a summarized view of all of the technical systems in an organization.

3. How to Set Up Data Gateways

3.1 Prerequisites

In order to implement the Data Gateways feature via the steps in this document, the following prerequisites must be satisfied.

  • Blue Prism v6.5 or later is installed
  • SQL Server Management Studio is installed
  • Administrative access to the Blue Prism database
  • Blue Prism instance is running as an Application Server

3.2 Setup Overview

Below are the steps required to configure the Data Gateways feature. By the end of these steps, Data Gateways will be configured such that Session Logs will be exported to a file on the desktop when a process runs.

  • Install the Data Gateways Engine
  • Enable the Data Gateways Process
  • Create a Database User for Data Gateways
    • Create a New Database User
    • Configure General User Settings
    • Configure User Mapping
    • Configure Securables
    • Configure SQL Server Instance Properties
    • Configure SQL Server TCP/IP Properties
  • Add the Database User to Blue Prism Credential Manager
  • Configure General Data Gateways Settings
  • Configure a Data Gateways Output
    • Create an Output
    • Restart Blue Prism Services
    • Run a Process

Step 1: Install the Data Gateways Engine

Data Gateways is not included with the standard Blue Prism installation, so it must be installed separately. The installer is located on the Blue Prism portal.

Select Product > Extras

01 Portal

For Blue Prism 6.5, download the Data Gateways v1.0 installer. For Blue Prism 6.6, download the Data Gateways v1.1 installer.

02 Data Gateways Engine Download

Run the downloaded installer to install the Data Gateways engine.

03 Data Gateways Engine Install

If the installation is successful, Data Gateways will appear in Control Panel, under the list of installed programs.

04 Control Panel

Step 2: Enable the Data Gateways Process

Navigate to and launch BPServer.exe which can be found in the following location:

C:\Program Files\Blue Prism Limited\Blue Prism Automate\BPServer.exe

05 BPServer

Click Edit...

Check the Enable Data Gateway Process box

Leave the default port as 8101

Click Save to commit the changes and close BPServer.exe

06 BPServer Edit

Step 3: Create a Database User for Data Gateways

Because Data Gateways works by pulling data out of the Blue Prism database, it needs a way to connect to that database. To do so, a user must be created in the Blue Prism database with the appropriate role and permissions to extract data. Data Gateways will use those user credentials to log in to the Blue Prism database and extract data.

In addition to creating a new user, this step will also cover some settings that need to be configured for the SQL Server instance to allow Data Gateways to connect reliably.

Step 3.1: Create a New Database User

Launch SQL Server Management Studio and connect to the database instance that contains the Blue Prism database as an administrative user

07 SSMS

On the left-hand-side Object Explorer menu, expand the Security folder

08 SSM Security

Right-click the Logins folder and select New Login

09 SSMS Login

Step 3.1.1: Configure General User Settings

Enter the following Login Name: Data Gateways BP Database SQL User

Select SQL Server Authentication

Create a password

Select the Default Database to be the Blue Prism database

10 SSMS General

Step 3.1.2: Configure User Mapping

On the left-hand menu, select User Mapping

Check the box for the Blue Prism database

11 SSMS User Mapping

Click the ellipses button in the Default Schema cell

Enter dbo in the text area and click OK to add the schema

12 SSMS Schema

On the Role Membership list, check the BPA_DataGatewaysEngine role

13 SSMS User Mapping Role

Step 3.1.3: Configure Securables

On the left-hand menu, select Securables

14 SSMS Securables

Click the Search... button and select "The server <Host>\<Instance>"

Click OK 15 SSMS Securables Add Objects

On the Permissions list, check the Grant box for the Connect SQL permission.

16 SSMS Securables Add Objects

Click OK to create the database user

Step 3.2: Configure SQL Server Instance Properties

To ensure this new user can connect to the database, the SQL Server instance must be configured to allow users to connect via SQL Server authentication.

Right-click the top-most SQL Server instance in the Object Explorer menu and select Properties

17 SSMS Properties

Click the Security listing on the left-hand menu

Under the Server Authentication panel, ensure that SQL Server and Windows Authentication mode is selected.

18 SSMS Server Authentication

Click OK to save changes.

Step 3.3: Configure SQL Server TCP/IP Properties

To ensure that Data Gateways can reliably connect to the Blue Prism database, the SQL Server instance must be configured to allow communication over port 1433.

Launch SQL Server Configuration Manager

19 SSCM

In the left-hand menu, expand the SQL Server Network Configuration entry and click the Protocols for <SQL Server Instance> entry

20 SSCM Protocols

In the right-hand menu, right-click and select Properties for the TCP/IP entry

21 SSCM Protocols Properties

Under the Protocol tab, ensure Enabled and Listen All are set to Yes

22 SSCM Listen All

Switch to the IP Addresses tab

For each IP Address listed in this menu, ensure that port 1433 is listed for each TCP Port entry

23 SSCM Ports

Click OK to save changes

Step 4: Add Database User to Blue Prism Credential Manager

To ensure Data Gateways knows the SQL Server username and password created in the previous step, it needs to be added to Blue Prism's Credential Manager. This is where the Data Gateways engine will look for the database user credentials.

Launch Blue Prism and sign in as an administrative user

24 Blue Prism

Navigate to System > Security > Credentials and click New

25 Blue Prism Credentials

Enter the following information on the form:

Name: Data Gateways BP Database SQL User

Type: Data Gateway Credential

Username: Data Gateways BP Database SQL User

Password: same password as entered when creating the user in SQL Server Management Studio

26 Blue Prism New Credential

Click OK to save the credentials

Step 5: Configure General Data Gateways Settings

This step will configure general settings that apply to the Data Gateways feature. This step does not configure any specific Data Gateways outputs yet.

If not already running, launch Blue Prism and sign in as an administrative user

Navigate to System > Data Gateways > Settings

27 Data Gateways Settings

Session Logs Panel

The first panel contains settings for how Session Log data should be handled with Data Gateways. The first checkbox, if left unchecked, will skip writing Session Logs to the Blue Prism database. This means Session Logs will only be available through a Data Gateways output.

The second checkbox, if checked, will allow Session Logs to be exported via Data Gateways. This does not commit Session Logs to any specific outputs yet, it just allows the user the option to include Session Logs as something that can be potentially exported.

Published Dashboards Panel

The second panel contains settings for Published Dashboards.

The checkbox, if left checked, will allow Published Dashboards to be exported via Data Gateways.

Published Dashboards are created and stored in Analytics > Published Dashboards

For each dashboard, set the Send Interval value (in minutes) to determine how frequently Data Gateways should export a refreshed set of Published Dashboard data to any Data Gateways outputs that are configured to receive Published Dashboards.

Data Gateways Processing Engine Panel

The third and final panel on the Data Gateways - Settings page can be used to configure how often (in seconds) the status of Data Gateways should be updated.

The status of Data Gateways can always be viewed in Control Room > Data Gateways

This page will display the current status of Data Gateways, if it is online, running, or if if has run into any errors.

Step 6: Configure a Data Gateways Output

The last step is to configure an output so Data Gateways can export some data. This guide will illustrate how to export a Session Log to a file in a folder on the desktop.

As multiple outputs are configured, they are written into a single configuration file within Blue Prism. This configuration file can be either edited manually, or it can be automatically managed by using the wizards.

Step 6.1: Create an Output

To create the first output, this guide will use the wizard.

If not already running, launch Blue Prism and sign in as an administrative user

Navigate to System > Data Gateways > Configuration and click Add New Gateway Output

28 Data Gateways Configuration

Enter an output name, such as: Session Logs - Desktop File

Leave the Output type as File

Enter a complete file path where the Session Log should be written, such as:

C:\Users\<User>\Desktop\SessionLogs\SessionLog_%25{+yyyy-MM-dd-hh-mm}.txt

File paths can include variable date formats, such as %25{+yyyy-MM-dd-hh-mm} so that the same file does not get overwritten with new Session Logs every time Data Gateways performs an export.

29 File Output

Click Next

Check the box for Session Logs

30 Output Data

The final screen displays the text that will be added to the main configuration file. This text identifies that Session Logs will be sent to the configured file path.

31 Output Config

Click Finish to complete the wizard and click OK on the message about restarting Data Gateways

Step 6.2: Restart Blue Prism Services

As a final step, this guide will illustrate how to run a process so that Data Gateways can export a Session Log to the file path that was configured in the previous step.

Before proceeding, it is important to restart the various Blue Prism services and processes to ensure a clean start.

Close Blue Prism

Launch Services.msc and stop the Blue Prism Server service if it is running.

32 Blue Prism Service Stop

Launch SQL Server Configuration Manager and right-click the SQL Server instance entry. Click Stop. Then right-click again and select Start to restart the SQL Server instance.

33 SSCM Restart

Close SQL Server Configuration Manager

In Services.msc, right-click and Start the Blue Prism Server service.

34 Blue Prism Service Start

Now, both Blue Prism Server and the SQL Server Instance should be running again.

Step 6.3: Run a Process

Launch Blue Prism and sign into the Server connection as an administrative user.

35 Blue Prism Launch

Navigate to Control Room > Data Gateways. The status should read: Starting

After a few seconds, the status should update to: Running which means Data Gateways is ready to process outputs.

36 Blue Prism Control Room

Navigate to Control Room > Session Management. Drag a Process to an available Runtime Resource and start the process.

37 Blue Prism Start Process

When the Process completes, a Session Log will be generated and written to a file on the desktop. Data Gateways has successfully exported a Session Log to a file location. Each process that runs will generate new Session Logs to the desktop in the same manner.

38 Session Log

From here, the Data Gateways configuration can be expanded upon to add more outputs.

For more advanced information about other Data Gateways outputs and filtering techniques, refer to the additional Data Gateways documentation on the Blue Prism Portal.