New OBIEE Sample App V305 (11.1.1.7) is released

The new OBIEE 11.1.1.7.0 Sample app V305 Virtual box image is available on OTN to download @ http://www.oracle.com/technetwork/middleware/bi-foundation/obiee-samples-167534.html.

Highlights of this Sample Application:

• Optimized deploy and scalability : 64b enterprise install, auto IP recognition
• BI .7 New Features : Front End – Back End
• BI + Essbase Integration : PA (Physical Layer Aggregation) spin-off essbase cubes, Security, Calc scripts, Writebacks
• BI + ORE Integration : Advanced ORE integration-interaction
• BI + Db Advanced Analytics : Advanced Analytics, NDM, Db Web Services, ODM
• BI + Endeca Integration : EID reading OBIEE, OBIEE navigating to EID
• BI + EPM/FR Integration : FR examples, EPM Companion Pack Deploy
• Other : BI custom functional interactions, Concurrency Simulation, Physical SQL generator

Thanks,
Shiva

 

New BI Apps 11.1.1.7.1 documentation (updated)

If anyone planning to install the new OBI Apps 11.1.1.7.1, Use the updated document from Oracle @ http://docs.oracle.com/cd/E38317_01/index.htm

Thanks,
Shiva

 

How to perform a full load in BI Apps 11.1.1.7.1?

In new BI APPS 11.1.1.7.1, the OOB tools (BI Application configuration Manager and Functional setup manager) are really helpful in terms of populating the Domain values automatically. And also it will provide a clear checklist of tasks that should be completed before doing a load. This is really an outstanding work from oracle to minimize the dependency on documentation to configure the BI Apps.

I have outlined the high-level steps to perform a BI Apps 11.1.17.1 Full load into warehouse.

In BIACM (BI Application Configuration Manager): Login to biacm (http://<hostname>:<port>/biacm) and follow these instructions:

1. Configure the Source System Connection Pool by choosing your Source Product Line and Version.

   

2. Select the offerings e.g. Oracle Project Analytics/ Finance Analytics/ etc…

   

3. Click on “Perform Functional Configurations” under Functional Configuration Task list and configure the Offerings in Functional Setup Manager (FSM). Here the FSM will provide us a checklist of the tasks we need to do like the list of parameters and other functional settings that we need to setup for a load.

   

4. And come back to biacm application and click on “Manage Load Plans” under “Load Plans” Tasks list to create the Load Plans. In BI APPS 11.1.1.7.1, we have a provision to separate the Loads into mini batch loads to minimize the applications/database downtimes. Generally, we will create one Domain Load Plan (SDE, SIL) and one Source Extract and Load (SDE, SIL and PLP). But it’s all depends on the Load strategy we define and will create different set of Load plans like SDE, SIL and PLP.

   Domain Load Plan (SDE, SIL): which does extracting the data from Source system and populate the configuration files to load data into

   

   Source Extract and Load (SDE, SIL and PLP):
   

   

5. Once we created the load plans, save them and generate the Load Plan details. Now click on Execute button to kick off the Load.

Once the Load plan is started, you can monitor the load with tasks by tasks status in ODI Console/ODI Studio. I would prefer the ODI studio as best practice to use to monitor the loads and troubleshooting the issues.

In ODI Studio:

After Load Plan started, Login to ODI Studio with biapps admin username and password, and navigate to Operator, to monitor the load plan with the steps (tasks) those are configured either in sequential or Parallel order for execution. Here you can see the overall Load Plan status by Tasks and also the summary of tasks execution to know how many are in the state of Done/Error/Running/Waiting etc..

We will troubleshoot the failed tasks and after issue fixes we have the ability to start that particular task individually. If any failed task ran successfully after fixes, The ODI will resume the load plan automatically to start next dependent (serial) tasks. Otherwise, we can also resume the Load plan with its “restart from the failure” option.

And also if we feel if the failed task is not a potential failure for the load, then we can “Mark as complete” it like DAC.

Upon Load Plan is successful, the ODI will load data into Warehouse and frees up the work tables where it was doing the transformations.

A sample Screen Shot of BI APPS 11.1.1.7.1 Oracle Project Analytics Dashboard after a full load using ODI 11g:

Have a great time folks…

 

Thanks,
Shiva

 

Oracle Data Integrator(ODI) Component Architecture

The Oracle Data Integrator platform integrates in the broader Fusion Middleware platform and becomes a key component of this stack. Oracle Data Integrator provides its run-time components as Java EE applications, enhanced to fully leverage the capabilities of the Oracle WebLogic Application Server. Oracle Data Integrator components includes the FMW exclusive features for Enterprise-Scale Deployments, high availability, scalability, and hardened security.

Oracle Data Integrator Component Architecture

ODI Repositories

The central component of the architecture is the Oracle Data Integrator Repository. It stores configuration information about the IT infrastructure, metadata of all applications, projects, scenarios, and the execution logs. The architecture of the repository is designed to allow several separated environments that exchange metadata and scenarios (for example: Development, Test, Maintenance and Production environments). The repository also acts as a version control system where objects are archived and assigned a version number.

The ODI Repository is composed of one Master Repository and several Work Repositories. Objects developed or configured through the user interfaces are stored in one of these repository types.

Master Repository:

There is usually only one master repository that stores the following information:

• Security information including users, profiles and rights for the ODI platform
• Topology information including technologies, server definitions, schemas, contexts, languages and so forth.
• Versioned and archived objects.

Work Repository:

The work repository is the one that contains actual developed objects. Several work repositories may coexist in the same ODI installation (for example, to have separate environments or to match a particular versioning life cycle). A Work Repository stores information for:

• Models, including schema definition, data stores structures and metadata, fields and columns definitions, data quality constraints, cross references, data lineage and so forth.
• Projects, including business rules, packages, procedures, folders, Knowledge Modules, variables and so forth.
• Scenario execution, including scenarios, scheduling information and logs.

When the Work Repository contains only the execution information (typically for production purposes), it is then called an Execution Repository.

ODI Studio and User Interfaces:

Administrators, Developers and Operators use the Oracle Data Integrator Studio to access the repositories. This Fusion Client Platform (FCP) based UI is used for administering the infrastructure (security and topology), reverse-engineering the metadata, developing projects, scheduling, operating and monitoring executions.

ODI Studio provides four Navigators for managing the different aspects and steps of an ODI integration project:

• Designer Navigator is used to design data integrity checks and to build transformations such as for example:
  • Automatic reverse-engineering of existing applications or databases
  • Graphical development and maintenance of transformation and integration interfaces
  • Visualization of data flows in the interfaces
  • Automatic documentation generation
  • Customization of the generated code




• Operator Navigator is the production management and monitoring tool. It is designed for IT production operators. Through Operator Navigator, you can manage your interface executions in the sessions, as well as the scenarios in production.




• Topology Navigator is used to manage the data describing the information system’s physical and logical architecture. Through Topology Navigator you can manage the topology of your information system, the technologies and their data types, the data servers linked to these technologies and the schemas they contain, the contexts, the languages and the agents, as well as the repositories. The site, machine, and data server descriptions will enable Oracle Data Integrator to execute the same integration interfaces in different physical environments.




• Security Navigator is the tool for managing the security information in Oracle Data Integrator. Through Security Navigator you can create users and profiles and assign user rights for methods (edit, delete, etc.) on generic objects (data server, data types, etc.), and fine-tune these rights on the object instances (Server 1, Server 2, and so forth).

ODI SDK:

Oracle Data Integrator also provides a Java API for performing all these run-time and design-time operations. This Oracle Data Integrator Software Development Kit (SDK) is available for standalone Java applications and application servers.

Run-Time Agent:

At design time, developers generate scenarios from the business rules that they have designed. The code of these scenarios is then retrieved from the repository by the Run-Time Agent. This agent then connects to the data servers and orchestrates the code execution on these servers. It retrieves the return codes and messages for the execution, as well as additional logging information – such as the number of processed records, execution time and so forth – in the Repository. The Agent comes in two different flavors:

• The Java EE Agent can be deployed as a web application and benefit from the features of an application server.
• The Standalone Agent runs in a simple Java Machine and can be deployed where needed to perform the integration flows.

Both these agents are multi-threaded java programs that support load balancing and can be distributed across the information system. This agent holds its own execution schedule which can be defined in Oracle Data Integrator, and can also be called from an external scheduler. It can also be invoked from a Java API or a web service interface.

Oracle Data Integrator Console

Business users (as well as developers, administrators and operators), can have read access to the repository, perform topology configuration and production operations through a web based UI called Oracle Data Integrator Console. This web application can be deployed in a Java EE application server such as Oracle WebLogic.

To manage and monitor the Java EE and Standalone Agents as well as the ODI Console, Oracle Data Integrator provides a new plug-in that integrates in Oracle Fusion Middleware Control Console.

Thanks,
Shiva

 

OBI Apps 11.1.1.7.1 installation and Configuration

Hello all, I am writing this post to provide the BI Apps 11.1.1.7.1 installation steps along with OBIEE 11.1.1.7.0.

If you didn’t install OBIEE 11.1.1.7.0 then follow all the steps from starting. Otherwise, skip the first 4 steps and continue. but make sure you have the weblogic 10.3.6 or upgrade to it, which is mandatory for ODI and BI Apps 11.1.1.7.1 products. The current BI APPS version is certified with just Oracle as Target db. So the Oracle db version should be 11.2.0.3 +. Because ODI is using JDBC drivers to connect to Oracle and will open multiple JDBC connections during the load. The DB  versions prior to 11.2.0.3 are not supporting multiple JDBC connections at a time. So Oracle 11.2.0.3+ are mandatory for successful ETL loads.

• Install JDK 1.6.X (JDK 1.7 is not certified by ODI 11.1.1.7)
• Install weblogic 10.3.6 (ODI 11.1.1.7 must be installed on WL 10.3.6)
• Run OBIEE 11.1.1.7.0 RCU utility to create BIPLATFORM and MDS schemas.
• Install and configure OBIEE 11.1.1.7
• Install ODI with All components and “Skip Repository Configuration” option.
• Run OBI Apps RCU to create schemas for:
  • ODI Master and Work Repositories (DEV_BIA_ODIREPO)
  • Oracle Business Applications Components (DEV_BIACOMP)
  • Oracle Business Analytics Datwarehouse (DEV_DW)
  • Update to MDS schema.
• Install and configure BI Apps 11.1.1.7.1. Basically, The biapps configuration is a process of extending existed weblogic domain(bifoundation_domain) with deploying new applications like biacm,odiconsole and etc…
• Apply FMW Platform Patches: – This is an important step to avoid the BI Apps configuration failure irrespective of what type OBIEE installation (software only / Enterprise) you did. Download “Oracle Fusion Middleware Platform Patches for Oracle Business Analytics Applications Suite” from the Oracle Business Intelligence Applications 11.1.1.7.1 media pack on Oracle Software Delivery Cloud.
  • D:\Middleware\Oracle_BI1\perl\bin> perl D:\Middleware\Oracle_BI1\biapps\tools\bin\APPLY_PATCHES.pl D:\Middleware\Oracle_BI1\biapps\tools\bin\apply_patches_import.txt
    
• Update BIACOMP Schema with ATGLite patch scripts:
  • sqlplus DEV_BIACOMP/<Password>
    
  • SQL> D:\Middleware\Oracle_BI1\sdf\DW\ATGPF\sql\fndtbs_11170_upg.sql
    
• Integrate ODI security with OPSS:
  • D:\Middleware\user_projects\domains\bifoundation_domain\bin>setDomainEnv.cmd
    
  • D:\Middleware\user_projects\domains\bifoundation_domain>java weblogic.WLST D:\Middleware\Oracle_BI1\bifoundation\install\createJPSArtifactsODI.py embedded –ADMIN_USER_NAME weblogic –DOMAIN_HOSTNAME localhost –DOMAIN_PORT 7001 –DOMAIN_HOME_PATH D:\Middleware\user_projects\domains\bifoundation_domain
• Setup ODI Studio to connect to BI Apps repository:
  
  • Open ODI Studio from windows Start à All Programs à Oracle à ODI Studio
    
  • In ODI Studio window, click on connect to Repository.
  • In Oracle Data Integrator Login window, Click on + icon to add a login credentials.
  • Provide all the details required as per the below screen.

  

  • Click Ok, and now you will see the Oracle Data Integrator Login window.
  • Login with the BI apps Admin username and password.
  • Upon Successful authentication, now we connected to BI Apps 11.1.1.7.1 Repository.
  • In ODI Studio, On Left hand window, you can see the ODI Designer, Operator, Topology and Security tab panes for easy navigation to perform the required tasks.
  • Click on Designer pane, and expand the Project list. Here you will see the BI Apps project folder and its mappings directories.

    

Have a happy time with BI Apps 11.1.1.7.1 with ODI J.

Thanks,
Shiva

 

BI Apps 11.1.1.7.1 Architecture Overview

“Oracle Business Intelligence Applications are complete, prebuilt BI solutions that deliver intuitive, role-based intelligence throughout an organization. These solutions enable organizations to gain more insight and greater value from a range of data sources and applications including Oracle E Business Suite, PeopleSoft, JD Edwards, Siebel, and third party systems.”

Oracle BI Applications are built on the Oracle BI Suite Enterprise Edition, a comprehensive and market leading BI platform. This enables organizations to realize the value of a packaged BI application, such as rapid deployment, lower TCO, and built-in best practices, or to build custom BI applications—all on one common BI foundation.”

The new BI Applications 11.1.1.7.1 has been completely re-architected to leverage the power of ODI with its E-L-T architecture, and has new Out of box tools to increase the productivity and lower down the TCO. The BI Apps 11.1.1.7.1 has significantly expanded with new analytical contents.

The BI Apps 11.1.1.7.1 components:

• Source Systems: The Oracle or non-oracle OLTP systems that holds the data for ETL into the Oracle Business analytics warehouse. This data becomes the underlying data for your analyses and includes customer information, inventory, sales, marketing, accounts, and other types of data you collect about your business.

• Oracle Business Intelligence Enterprise Edition: Oracle Business Intelligence Enterprise Edition (Oracle BI EE) is a business intelligence platform that is used to access and present data in easy-to-understand formats, such as tables, graphs and Scorecards/KPIs.

• Oracle Business Analytics Warehouse: The Oracle Business Analytics Warehouse is a modular enterprise-wide data warehouse data model with conformed dimensions for data integrated from multiple sources. The Oracle Business Analytics Warehouse supports the analytical requirements of Oracle Business Intelligence Applications.

• Oracle BI Applications Configuration Manager (BIACM) and Functional Setup Manager (FSM):  This is similar to DAC tool. But unlike DAC, The BIACM and FSM have great features and functionality in BI APPS 11.1.1.7.1 to simplify the configurations process, minimize the manual steps and configurations, and to have a guided list of configurations steps and checklists.  That reduces the dependency on documentation….
  •  Oracle BI Applications Configuration Manager (BIACM):  The BIACM is a web based UI application for setting up and maintaining an Oracle Business Intelligence Applications environment. Oracle BI Applications Configuration Manager works in conjunction with Functional Setup Manager to provide guided Tasks to configure Offerings and Functional Areas.
  • Functional Setup Manager (FSM): The FSM is also a web based UI application which enables you to manage and perform functional configuration tasks and to deploy the Offering and Functional Areas that were selected. Functional Setup Manager generates a list of configuration tasks for the deployed areas, which you can assign to different functional developers and monitor the implementation status.

• Oracle Data Integrator: Oracle BI Applications uses Oracle Data Integrator (ODI) as its data integration platform. Oracle Data Integrator is a comprehensive data integration platform that covers all data integration requirements: from high-volume, high-performance batch loads, to event-driven, trickle-feed integration processes, to SOA-enabled data services with its innovative E-LT Architecture.  

  E-LT is an innovative approach to extracting, loading and Transforming data. Typically ETL application vendors have relied on costly heavyweight; mid-tier server to perform the transformations required when moving large volumes of data around the enterprise. ODI delivers unique next-generation, Extract Load and Transform (E-LT) technology that improves performance and reduces data integration costs, even across heterogeneous systems by pushing the processing required down to the typically large and powerful database servers already in place within the enterprise. 

  In an Oracle BI Applications environment, ODI works in conjunction with Oracle BI Applications Configuration Manager, which provides a user interface for managing load plans as well as access to ODI Console, the web-based console that enables you to view objects in the ODI repository.

• Oracle GoldenGate: Oracle GoldenGate is an optional product. Oracle GoldenGate is a data replication tool leveraged to create a replicated OLTP schema. This replicated OLTP schema is used as a Source- Dependent Schema during ETL.

A logical Overview of BI Apps 11.1.1.7.1:

Oracle BI Applications use all of the components depicted in above figure to deliver information specific to your business needs. The source system (OLTP) data is loaded into the Business Analytics Warehouse Database (OLAP) and it is transformed using the ODI mappings that are stored in the BI Applications Repository Database. This database contains the repository information for BI Applications ODI repository and the BI Applications Components Repository.

Optionally, you can leverage Oracle GoldenGate to replicate your OLTPs data and then to do Source – Dependent load. This helps us to least dependent on the source system availability for ELT operations.

 

 

Thanks,
Shiva Molabanti

 

OBIEE Impact Analysis – What if I delete a column from RPD, what are the reports that are being affected? – Part 2

In my recent post, I have shown how to gather the RPD and Webcatalog data required for data lineage. And now we will build an OBIEE rpd with these OBIEE Web Catalog and Repository CSV files by creating ODBC connections or we can import the CSV files data into a database table using any import tools. For example, I am using SQL developer to import data from CSV files into a database table. I used SQL developer excellent feature to import data from CSV files.  This is very simple, just expand the database and right click on the tables. Then select Import Data.

Select the CSV file to import.

Define the CSV file properties, like format type = delimited and delimiter = tab.

Proceed to next wizards. This will create a table in specified Oracle DB and imports the data into it. Repeat the same steps for RPD_Data.csv files to load into other table.

Now import these tables into OBIEE RPD and build a repository with all 3 layers.

I have used the following business logic to join the Catalog and RPD data. (The table and column names are dependent on what you have customized)

Catalog. Subject Area = RPD. Subject Area

RPD.Presentation Table = Catalog. Table

Catalog.Presentation Column = RPD.Column

Deploy this RPD and configure an empty webcatalog (or you can use existed one) through Oracle Fusion Middleware EM and restart all the OBIEE services.

Now we are ready with the data lineage information.  Just login to OBI Presentation Services and create the necessary reports to know about reports statistics and their columns definition all the way from physical sources.

For example,

Question 1: What are the physical columns used by a report?

Ok well, I’ll create a report with the required columns to answer this question (See the report below).

From above report I can answer, what are the columns the report is made up with, what is the definition of a metric and what is the source of that particular column.  And the above report also tells me what happens if I delete a particular column from RPD like what are the reports that are being impacted?

Question 2:  How many reports have been developed and who?

Create a report and answer there are 39 total reports and ownership details are below.

Like this we can create the reports to answer some crazy questions about column lineage.

Hope this helps you all..

 

Thanks,
Shiva