Sunday, 25 September 2016

EPM and ORDS – Part 1

Over the last year or so I have focused on the different Web Services options available in EPM products and paid particular interest to REST which in my opinion is the clear winner over the SOAP protocol and the standard for the future.

In the ever changing world of EPM the need for REST Web Services is becoming more apparent, you only have to take a look at EPM Cloud and see that REST is the core for automating processes and moving around metadata and data.

Oracle are having to move quickly to implement more and more RESTful based functionality with the rise of their cloud offerings, Oracle spent so much time bringing EPM products together that they didn’t really focus on Web Services when they should have but now that is all having to change.

Today I am going to continue with the REST theme and look at Oracle REST Data Services (ORDS) which I believe can help provide solutions to common situations that you encounter with EPM products.

So what is ORDS:

"Oracle REST Data Services (ORDS) makes it easy to develop modern REST interfaces for relational data in the Oracle Database and now, with ORDS 3.0, the Oracle Database 12c JSON Document Store and Oracle NoSQL Database. ORDS is available both as an Oracle Database Cloud Service and on premise."


So in basic terms ORDS is a web application that allows you to execute REST resources against Oracle databases.

You see there is a catch that this is not going to be everybody as ORDS will only run against Oracle databases, the good news though is that if you are licensed for Oracle database then you are also licensed to use ORDS.

As the majority of EPM products store information in a relational database then being able to simply make a call over HTTP to return something meaningful is only going to be a good thing.

I have lost count the number of posts I have seen or questions that have been asked about running SQL against the EPM repositories and this is where ORDS can help put some control over it and at the same time take away the need to access the database directly, we all know how much DBAs love giving out access to users.

Wouldn’t it be great if you could deliver the results from all those SQL queries that you have been asked to run without having to develop some cumbersome process and also provide access restrictions using inbuilt security options, well hopefully I am going to show how this can easily be done with ORDS and look at some examples to everyday EPM scenarios.

It may mean nothing to you at the moment but after seeing some examples of where ORDS can help I am sure you will be won over.

I am going to start out with going through the installation which to be honest is not complex but will need some discussions with your DBA as to get ORDS configured you will need administrative database privileges.

First of you all you need to download ORDS but don’t worry it is not a monster of an application and weighs in at less than 60mb.

The version of ORDS I am using is 3.0.6 which is the latest at the time of writing this post

As with any product there are system requirements that need to be met and for this version of ORDS they are:
  • Oracle Database (Enterprise Edition, Standard Edition or Standard Edition One) release 11.1 or later, or Oracle Database 11g Release 2 Express Edition.
  • Java JDK 1.7 or later.
ORDS requires a Java EE application server and the following are currently supported:
  • Oracle WebLogic Server - 11g Release 1 (10.3.6) or later
  • GlassFish Server - Release 3.1.2 or later
  • Apache Tomcat - Release 7.0.56 or later
There is also the option of running ORDS in standalone mode which does not require a Java EE application server but this is only suitable for develop use so it is required to go down the supported application server route.

As ORDS is an Oracle product then naturally WebLogic Server is supported which you may think great as I already have WebLogic Server installed as part of EPM, not so fast, first of all it is only limited-use license that is included with EPM which means you can’t deploy ORDS into your existing EPM domain, ORDS also requires Java JDK 1.7 or later and as we all know EPM is so far behind with the version of Java and is still running with version 1.6

There is no stopping you if you are fully licensed for WebLogic Server but you would need a separate install that is running a supported JDK version.

I have chosen to go down the Apache Tomcat route which to be honest is extremely simple to use and get up and running with.

A quick overview, download, run the installer, select the type of install and components which can be left as default.


Select the ports to run Tomcat on, I changed from port 8080 to 8020 as it was already being used on the machine I was installing on.


Select the supported Java location.


Select the location to install Tomcat to.


And for standard functionality that is pretty much all there is to it.

So back to ORDS, once downloaded extract to a desired location.


ORDS can be installed and configured through command line or with SQL Developer, I am going to use command line but in the next part I will introduce SQL Developer to make life easy administrating ORDS.

There are two options for installing ORDS, simple which uses a parameter file and advanced using command line prompts.

I have chosen advanced but it is hardly advanced once you see it.

Before starting out you will need an Oracle database user with SYSDBA privileges which can be temporary as it is only required for the configuration.

To understand what database users and privileges the configuration creates then refer to the documentation which definitely will be useful in discussions with your DBA.

To start installing and configuring then make sure you are using a supported JDK and run the following command:

java -jar ords.war install advanced

The first part of the configuration is to enter the location to store the ORDS configuration data and the Oracle database connection information.


Next keep the default to create the REST Data Services schema as it has not been created yet.

Enter a password for the ORDS_PUBLIC_USER.


The ORDS_PUBLIC_USER is the database user that ORDS uses to communicate with the database and invoke RESTful services in the ORDS enabled Oracle schemas.

Next steps are to enter the details for the SYSDBA user and provide tablespace information or keep the defaults.


The final input is to select 2 as we are not using Application Express (APEX)

ORDS was formerly known as APEX listener and part of Application Express, ORDS can now be independent of APEX which is how I will be using it.


That is the configuration done, pretty simple.

A configuration file ords_params.properties will have been created.



To validate the configuration, you can run the following.


To deploy ORDS in tomcat is once again extremely easy.

Move ords.war into the Tomcat webapps folder.


Tomcat will then automatically deploy the ORDS web application


To test the deployment was successful go to the following URL for ORDS:

http://<hostname>:<port>/ords/

Do not worry about the 404 Not Found message as this is to be expected when ORDS is not being used with APEX.


As a final step I added the ORDS web application to the OHS EPM configuration file.


This allow ORDS to be accessed through OHS so end users can go through the same route as they would to access EPM products and no need to start worrying about opening ports and additional URLS to manage.


I am going leave it there for today now that ORDS is up and running, in the next post I will start looking at administrating ORDS and creating REST resources against EPM database repositories.

Monday, 22 August 2016

Smart View and Essbase Runtime Substitution Variables

Back in August 2015 when Smart View 11.1.2.5.500 was released the readme had reference to new functionality for runtime substitution variables, the readme contained the following information:

"In Smart View, you can now execute Oracle Essbase calculations that, when launched, prompt you to enter variable information, called runtime prompts. Calculation scripts and runtime prompts are created by your Essbase administrator for your specific system.

Essbase administrators define calculation scripts so that Smart View users can work with runtime prompts. For information on setting up your calculation scripts with runtime prompts, see the Oracle Essbase Database Administrator's Guide, “Using Calculation Scripts with Runtime Substitution Variables in Smart View.”

After a while it was confirmed this functionality was not yet available and the information was subsequently removed from the readme.

The Essbase 11.1.2.4.010 patch readme contained reference to runtime substitution variables in the documentation updates section:

 “Correction: Using Runtime Substitution Variables in Calculation Scripts Run in Smart View
The following information updates the “Using Runtime Substitution Variables in Calculation Scripts Run in Smart View” section of the Oracle Essbase Database Administrator's Guide for 11.1.2.4”

I have searched for the section in the 11.1.2.4 Essbase documentation and could not see it but it is a different story if you look at the Essbase 12c documentation.

At the time of writing this the latest patch release of Essbase 11.1.2.4.011 and the functionality still doesn’t look available which I am sure will change in a future patch.

Currently Essbase 12c is only available in OBIEE 12c and I wrote a detailed post about it last year looking at the changes.

So are runtime substitution variables available in Smart View with Essbase 12c, well this would be a short post if they were not :)

It is worth pointing out that it doesn’t look like it is the version of Smart View or Essbase that is holding back the functionality in 11.1.2.4, it is code differences in Provider Services that are making the difference.

In this post I thought I would look at a few extremely simple examples of using the runtime substitution variable functionality in Smart View.

The examples are using Smart View 11.1.2.5.600 and Essbase/Provider Services 12.2.2.1, there is no EAS in 12c but the good news that EAS 11.1.2.4.008+ is now supported with Essbase 12.2.2.1 which makes life much easier, you are still under the constraints of OBIEE security but at least you can use the majority of functionality once the Essbase 12c server has been added in the EAS console.

In OBIEE 12c you can still use a Smart View connection URL and the only difference is the default port will be 9502.


The same goes for private connections:


The default Essbase cluster name is “bi_cluster


Before going any further with Smart View we need a calculation script to run that includes runtime substitution variables.

The syntax for using Smart View runtime substitution variables in a calc script is:

SET RUNTIMESUBVARS
{
   rtsv = POV
   <RTSV_HINT>
      <svLaunch>
         <description>rtsv_description</description>
         <type>member | string | number</type>
         <dimension>dimName</dimension>
         <choice>single | multiple</choice>
         <allowMissing>true | false</allowMissing>
      </svLaunch>
   </RTSV_HINT>;
};

The different options within the tags will become clearer as I go through some examples.

Let us start with one runtime sub var.


I will breakdown the above definition of the sub var
.
“Market = POV” - A variable called “Market” has been defined which is set to POV, when set to point of view (POV) it basically means it will pick up information from the current member(s) within the grid in Excel.

<RTSV_HINT> and <svLaunch> are a requirement of using runtime sub vars in Smart View.

<description> just describes the runtime sub var and is shown in the runtime prompts section in Smart View.

<type> can be set as member, string or number, in this example I want the sub var to be a member.

<dimension> this is the dimension that the sub var is defined against which in my example will be the “Market” dimension

<choice> has been set to single to assign one member to the sub var, the other option is multiple.

<alllowMising> the definition of this is

“Specifies whether to allow or suppress data cells for which no data exists in the database”


To be honest I have not seen this actually make any difference whatever it is set to, I am probably missing something obvious.

So in summary all that definition means is that the current Market member in the Excel grid will be picked up and assigned to the “Market” sub var, the sub var is then used in the FIX.

It makes more sense with a demo of it in action, I have created a simple retrieve in Excel.


The market dimension member in the retrieve is “Florida” which should be picked up and assigned to the sub var.

Select “Calculate” to bring up the calculation scripts window.


Once the calc script I have created has been selected a “Runtime Prompts” section is displayed.


The description of the sub var defined in the calc script is shown and the member “Florida” has been correctly picked up from the grid.


The calc script was successfully run and a quick check of the Essbase application log confirms the member “Florida” was assigned to the sub var and used in the fix.

Aggregating [Product(All members)] with fixed members [Market(Florida); Scenario(Actual)]

A refresh of the sheet shows the product dimension has been aggregated.


The runtime sub var definition can be easily expanded and this time I have added a Scenario variable into the script.


It is probably worth pointing out that if you do validate the script in the console you will be hit with an error.


The log will contain:

Error: 1200315 Error parsing formula for [POV] (line 5): invalid object type

This error can be ignored but one to watch out for if you need to validate a complex script.

Anyway, back to the example, if I run the calc on the same grid in Smart View this time the current Scenario is picked up.


After running the calc the log verifies the sub var was correctly assigned.

Aggregating [ Product(All members)] with fixed members [Market(Florida); Scenario(Actual)]

It is working fine for single members but how about multiple members.

This time I updated the definition to set choice to multiple for the Market sub var.


The Excel grid now includes two Market members.


Selecting the calc script now displays a different message in the runtime prompt box.


The message does not quite it on and expanding the window makes no difference but the full message is

“Market Dimension Members on Row/Col/POV will be used.”

After launching the calc the log confirms that both Market members were successfully assigned to the sub var.

Aggregating [ Product(All members)] with fixed members [Market(New York, Florida); Scenario(Actual)]

So what happens if choice is set as “single” but there are multiple members in the grid.

I set “choice” back to “single


In Excel I have purposely not selected any Market member cell.


Select the calc script and the following warning is displayed.


The calculation script window is still displayed with the Market runtime prompt empty which cannot be changed so the only option is to close the window.


This time I have selected a Market member cell.


No warning this time and the selected member is assigned.


Ok, moving on to some of the other options available in the runtime sub var definition.

If you don’t want to use the POV method and manually enter the value for a sub var then the “string” type can be used.


In the above example “type” has been set to string and POV updated to a Scenario member.

Interesting that in the patch readme the documentation update states:

“The value of the runtime substitution variable must be set to POV; it cannot be set to a member name.”

The way I read that is what I am doing now cannot be done or maybe as usual it is some cryptic message that can only be understood when all the planets align.

The “choice” value will be ignored and can be removed as it is only valid when “type” is set as “member

The calc script was run again and now the runtime prompt is populated with member set in the sub var definition.


The runtime prompt can be updated or left as the default.


A comma separated list of members is also a valid input.

A quick check of the log verifies the string type is working.

Aggregating [ Product(All members)] with fixed members [Market(Florida); Scenario(Actual, Budget)

If member selection is required, then the “type” can be set back to “member


The runtime prompt will now be enabled for member selection.


As “choice” has been set to single only one member can be selected.


As expected the member is correctly assigned to the sub var in the calc script.

Aggregating [Product(All members)] with fixed members [Market(Florida); Scenario(Actual)]

To select multiple members then “choice” can be updated to “multiple


The member selector will now accept multiple members.


Aggregating [ Product(All members)] with fixed members [Market(Florida); Scenario(Actual, Budget)]

On to the final “type” which is number


In the above example the sub var was defined as “SalesRate” and given a default value of 1.2, “type” was set as “number” and the “SalesRate” sub var is then used in a calculation.

The following Smart View grid was set up in Excel.


This time when selecting the calc script a new runtime prompt is displayed with the “SalesRate” sub var.


The default value can be overridden by entering a new value.

The calc ran successfully.

Calculating [ Measures(Sales)] with fixed members [Product(100-10); Market(New York, Florida); Scenario(Budget)]

A refresh of the data and confirmation that Sales data has been multiplied by 1.2.


If you are not aware it is possible to include logging of the runtime variable values in the application log by using the ENABLERTSVLOGGING configuration setting.

I added the following to the Essbase configuration file so runtime sub vars would be logged for the Sample application.

ENABLERTSVLOGGING Sample TRUE

After running the calc script again the log contained the runtime sub var details.

Executing calc script [MTRTP1.csc] with runtime substitution variable { Market = "Florida","New York" }

Executing calc script [MTRTP1.csc] with runtime substitution variable { Scenario = Budget }

Executing calc script [MTRTP1.csc] with runtime substitution variable { SalesRate = 1.2 }

I did notice a bug in Smart View when using multiple members with the sub var set as POV and also having a sub var defined as a member.


There is no problem when the member selection is not selected so the defaults are kept.


If you do open the member selection and then run the calc the following error will be generated.


The Essbase application log highlights the issue as instead of passing the members from the grid through it has sent the default message “Market Dimension Members on Row/Col/POV will be used”

Invalid Calc Script syntax [
FIX(Market Dimension Members on Row/Col/POV will be used, "Budget","Actual")
AGG(Product);
ENDFIX;]

Smart View basically sends XML to provider services and by capturing what is being sent from Smart View you can see it is incorrect


It is an issue with Smart View and not provider services though I am sure it will be fixed in a future Smart View release.

Well I think I am going to leave it there for today, once the functionality is available in 11.1.2.4 I will update this post to reflect that.