Vizlib Server Guides - Loop & Reduce Tasks

In the Writeback Settings section of the Vizlib Management Console (VMC), the Loop & Reduce Tasks settings use loop values to break down large applications (apps) into smaller ones (reduced apps). This guide takes you through an example of how to create and execute a loop and reduce task. 

TABLE OF CONTENTS

• Overview / Benefits
• Updating the design of reduced applications
• Scheduled (nightly) reloads
• Reload after writeback
• Checklist
• Creating a Task
• Security Rules Wizard
• Executing Tasks
• Demo Apps 
• Data Load Scripts and Data connections
• Destination

Overview / Benefits

Loop & Reduce Tasks allows you to split a large app into reduced apps based on loop values. Once 

the reduced apps are created, you can create security rules to make sure users only have permission to see the data they should. I.e. each app is split to contain only the data they can access. As a feature, it is a good complement to section access in Vizlib Writeback and can be used to deliver several benefits. 

The examples in this article use qvd-files for simplicity but of course any of the available data sources can be used.

• Smaller applications have a quicker response and reload times.
• Selections are not released on reload (Section Access will otherwise typically force selections to be released)
• Reduces the impact of concurrency challenges for Vizlib Writeback Table and Vizlib Input Form.

Figure 1: Executing Loop & Reduce Tasks, loop values Red, Green and Blue.

There are important things to consider when using Loop & Reduce Tasks. (If you have previously used QlikView Loop and Reduce this is similar but there are key differences). 

• Loop & Reduce Tasks only needs to be executed for deployment changes or creating reduced apps.
• Loop values are defined in the task (QlikView defined them from a dimension in the app).
• Reduced apps can be reloaded individually. 
• Bookmarks and user objects are persistent (copied and applied in the reduced app).
• We recommend using Binary reload with Loop & Reduce Tasks for quicker reloads. You can find out more about binary loads here. 
• Reduction logic needs to be added to the data load script of the source app.
• The reduced apps naming convention is the document title of the source app with ' - ' and the loop value appended at the end. E.g. Concurrency Demo - Green.

These features and differences are designed to give app developers more flexibility in developing and organizing access to specific features and improving app performance.

The Loop & Reduce Task is executed in the VMC to create the reduced apps for the first time and when changes to the source app needs to be deployed to the reduced apps. 

Updating the design of reduced applications

When changes are made to the design of the application they only need to be implemented in one place and then changes are propagated to the reduced apps.

Follow these steps to deploy updates to the reduced apps.

1. Duplicate the source app
2. Make changes to the duplicate source app
3. Publish the duplicated source app and replace the original source app
4. Execute Loop & Reduce task to update the reduced apps.

This will update the reduced app with the changes in the source apps and keeping user objects in the reduced apps such as bookmarks and user created sheets, since the reduced apps are replaced and not deleted when the task is executed (Figure 2).

Figure 2: Deploying changes. (Note how version number changes in the reduced apps based on the updated source app.)

Scheduled (nightly) reloads

Setup reload tasks for the Binary data source and create reload tasks for the reduce apps that should be triggered once the Binary data source task have finished successfully. Since the reduction logic is inside the load script of the reduced apps they can be reloaded with standard Qlik Sense reload tasks (Figure 3). The data connection to the Binary data source should be setup in a secure way will make the data secured. Manipulating the load script will not get users access to data that they should not have access to, because they do not have access to the data connection to the Binary data source. Similar to how section access, but access is configured with security rules in the QMC and security rules and destination config in the VMC.

A video guide to setting up binary loads is found here.

Figure 3: Nightly reloads of the reduced apps. Quick binary update of the larger data volumes and then the (typically smaller) data from each writeback is added.

Reload after writeback

When writeback is executed the reduced app where the writeback was initiated will be reloaded quickly with the source data from the Binary data source and the updated writeback data from the loop value qvd (Figure 4). This can be achieved with by configuring the destination used by the apps with dynamic tags for reload app and filename. See Destination section for more information .

Figure 4: Reload after Writeback of Green.qvd. 

Back to Contents

Checklist

Before you start, there are a few tasks you'll need to complete, we've listed them here for you. 

• QMC  - You'll need access to the Qlik Management Console (QMC), with permissions to upload apps and create/edit custom properties. 
• VMC - You'll need access to the VMC, with permissions to create and edit Security Rules. 
• [Optional] Demo App - Download the app QVF files attached to this article and upload them to your Qlik Sense environment using the QMC. You can also find an example app in our Demo Apps Gallery. 
  

Back to Contents

Creating a Task

To start working with Loop & Reduce Tasks, you'll need to log in to the VMC. 

Note: If you see the icon, click or hover to see more information about the feature or field.

• Navigate to Writeback Settings>Loop & Reduce Tasks (Figure 5), then click Create task when the page opens.

Figure 5: Loop & Reduce Tasks

When the Create Loop and Reduce Task screen opens (Figure 6) you'll see several empty fields where you'll need to enter information. You can complete the form by following the steps in the numbered list. 

Figure 6: Create Loop and Reduce Task

1. Enter a Task Title and check the Access and Security you need (Public means every VMC user will have access). 
2. In Select App to deploy in Stream, choose the App you're going to deploy (Concurrency Demo in this example) by typing the name and selecting from the predictive list.
3. Repeat step 2 for the deployment Stream (Concurrency Users). 
4. In the Loop section, enter a value in Add new loop value, then click to save it. HINT - it's possible to copy and paste multiple values from e.g. Excel into this input.
5. Click Security rules wizard to configure your security rules for the task (these instructions are listed in the Security Rules Wizard section of the article).
6. Enter a User and Password in Rest basic authentication.
7. Click Create Task when you've completed all these steps.

Back to Contents

Security Rules Wizard

Important: Modifying security rules can grant users permissions and access to data which they should not have. Always remember to verify any changes to a security rule in the Audit window of the QMC!

When the Security rules wizard screen opens (Figure 7) you'll be able to configure and modify security rules which grant access to the reduced apps. 

Note: You'll need to be logged into the QMC for this section.

Figure 7: Security Rules Wizard

1. In Select a User Attribute, select the User attribute you need to determine access to the app (e.g. Department or Group). The attribute value must be equal to the loop value. If you can't find a suitable attribute in your user directory, you can create a new custom property in the QMC. 

2. In App security rule, click Copy app security rule (from VMC) and open the Security Rules menu in the QMC.

• Click Create new to open the security rules screen, then select App Access from the Create Rule from Template dropdown (Figure 8).

Figure 8: App Security Rule

• Enter a Name for the security rule (this field is mandatory).
• Paste the copied app security rule into the Conditions text area in the Advanced section. 
• Click Preview to check the rule is set up correctly. It should be displayed in the audit window to the right of the settings panel. 
• Click Apply to save the rule.

3. The QMC default app access rule grants all users read access to all apps in every stream. For loop and reduce tasks, the rule must be modified to exclude the stream for the end-users reduced apps, and for that you can use Modify Stream security rule. 

• Click Copy modify stream security rule, and open the Security Rules menu in the QMC. 
• Open the default app access rule for editing. 
• Paste the copied stream security rule into the Conditions text area in the Advanced section and click Validate rule (same position as Figure 8).
• Click Preview to check the rule is set up correctly. It should be displayed in the audit window to the right of the settings panel. 
• Click Apply to save the rule.

Note: If you are not using the default app access rule you need to edit your security rules manually. 

Back to Contents

Executing Tasks

Once the task has been created and saved, you'll be able to manage it in the VMC (Figure 9). When you click on the task, you'll see a range of options appear - Execute task, Edit, Duplicate, Delete and Transfer Ownership.

Figure 9: Execute Task

When you click Execute task, you'll receive status updates on the task's progress before receiving a confirmation that the task has been successfully completed (Figure 10).

Figure 10: Loop & Reduce Task Execution Status

Back to Contents

Demo Apps 

The demo apps attached to this article are based on the Collaborative Forecast demo application. BINARY - Vizlib Collaborative Forecasts.qvf contains the data for the example. The Concurrency Demo.qvf performs a binary reload from the BINARY - Vizlib Collaborative Forecasts.qvf (you can find out more about binary loads here). 

Both these applications should be published to a stream (e.g. Concurrency - Binary & Dev) and then a separate stream (e.g. Concurrency Users) should be created for the end-users where the reduced apps are published. Figure 11 shows an example reduced app, where you can see the loop value 100 displayed as the suffix after the loop and reduce task have executed successfully.

Figure 11: Reduced (End-user) App Example

Back to Contents

Data Load Scripts and Data connections

This section contains examples of the data load scripts which need to be used with the demo apps.

The load script for Concurrency Demo.qvf starts with a Binary load of the source data (Figure 12). Create a new data connection with the name Concurrency BINARY that points to the BINARY - Vizlib Collaborative Forecasts.qvf (you can find out more about how to set up binary loads here). 

Figure 12: Binary Reload

Create a folder data connection with the name ConcurrencyData where the demo app will store and load its data. QVD files and folders will be created on reload of the demo apps. 

NOTE! If you want to use a data connection with a different name, you need to update the DataConnectionName variable in the README section of both applications (Figure 13).

Figure 13: ConcurrencyData data connection in README section of data load editor.

The reduction of data is configured in the load script of the source application (Concurrency Demo.qvf) and reduces/filter the data model when the reduced apps are reload. In the demo apps we use the app suffix (loop value) and filter the binary data on the Cost Center field. The FilterDataOnGroup sub-routine filters the Transactions and CostCenters tables based on the _Group equal to Cost Center field (Figure 14).

Figure 14: Filter Data Sub-routine

The Set Group Variable section (Figure 15) contains logic on how to extract the loop value from the Document Title. 

Figure 15: Loop & Reduce

The LOOP & REDUCE section (Figure 16) contains logic to verify the document name and loop values. If they are not correct or a value is not defined according to the If statement, all the data tables will be dropped. If the loop value is valid then the FilterDataOnGroup subroutine is called with the loop value as an argument.

Figure 16: Loop & Reduce

Destination

Create a new Writeback Destination in the VMC and fill it out as Figure 17.

Figure 17: ConcurrencyData Destination

1. For App, use the dynamic tag APPID using the plus icon. This allows the destination to be used by all the reduced apps and only reloads the app that is doing the writeback.
2. For Subfolder(s), type Groups/ and insert the dynamic tag APP.SUFFIX using the plus icon.
3. Enable File name set by the extension.
4. Add these File names to the File Name Whitelist: Approvers, CurrentUser, Forecasts, ForecastTransposed, ForecastUnderReview, Forecasters, Settings, Variables. Or copy all the values from the LoopValues.xlsx attached to this article and paste them in Add new File Name input to add them all at once.

Note: If you want to use a destination with a different id, you need to update the VizlibServerDestination variable in the README section of Concurrency Demo.qvf (Figure 18 & 19)

Figure 18: Vizlib Server Destination variable in README section of data load editor. 

Figure 19: Writeback Tables and Input forms configured Writeback Destination using VizlibServerDestination variable as Expression

To continue configuring the demo apps, create a new Loop & Reduce task following the steps here Creating a Task and execute it to create the reduced apps.

Back to Contents