[edit] See this post too [/edit] On an OBIEE server run [sourcecode language=“bash”] nohup obiee/systemsmanagement/runagent.sh & [/sourcecode] and then run jconsole (make sure you’ve set the DISPLAY first if you’re running it from UNIX). NB: if you don’t have jconsole in your path you can search for it: [sourcecode language=“bash”] $whereis jconsole jconsole: /opt/java1.5/bin/jconsole /opt/java6/bin/jconsole</span> [/sourcecode] You should find it under your java/bin directory

You should get this kind of connection dialog: Click connect, and the console will launch. From here click on the MBeans tab, where you’ve got access to performance and configuration data

I’m working on a scripted load test for OBIEE using nqcmd to run reports multiple times. I hit this interesting issue.

Cut and pasting the logical SQL that Presentation Services sends to BI Server from Manage Sessions -> Statement, I kept getting this error when I ran it through nqcmd:

[10058][State: S1000] [NQODBC] [SQL_STATE: S1000] [nQSError: 10058] A general error has occurred.
[nQSError: 27005] Unresolved column: “Natural Account (COA)”.“Account Parent1 Code”.
Statement preparation failed

It’s possible to change the error pages served up by OAS/Apache by using the ErrorDocument directive. This is widely documented.

However, to get this to take effect in an oc4j application (such as analytics) you need to change mod_oc4j.conf too.
(I found this out from this post here)

Take backups of httpd.conf and mod_oc4j.conf, and then edit them as follows:

In httpd.conf add:
ErrorDocument 500 /500.html
where /500.html is a relative path to your custom document

New releases this week - 10.1.3.4.1 of OBIEE, and 7.9.6 of OBIA.

After a bit of scrabbling around found:

“The 10.1.3.4.1 release of the Oracle Business Intelligence Enterprise Edition introduces no new features.”

Though there are some new bits and pieces for Publisher

For 7.9.6 OBIA, there’s no New Features document :(
By piecing together the “What’s New in This Release” for each document you can build up a picture (eg in the Upgrade Guide there’s reference to changes for the doc relating to a new version of Informatica), but it would be nice to have it all in one place:

We’ve got a setup of two OAS/Presentation Services boxes and two BI Server boxes, with load balancing/failover throughout.
The Load Balancing of the web requests is being done through a separate bit of kit, an F5 BIG-IP load balancer. This directs the requests at the two OAS servers.

The problem we have is that by default OAS serves HTTP on port 7777, but the F5 is using port 80. A request for our load balanced URL: http://bi.mycompany.com/analytics/ barfs out with

Why does Google more often than not return the korean (kr) version of OTN forums for a match, but not english?

If I search Google for “Send notification:” “oracle.ons.Notification” the top hit is for kr.forums.oracle.com/forums/thread.jspa?threadID=650662

If you strip the kr from the URL you get this, which makes more sense given the locality when doing the search in Google.

Only mildly irritating, but odd behaviour nonetheless

I noticed that the j2ee server.log file was filling up with these entries:

oracle.ons.Notification@afba5d
09/04/02 10:15:12.207 Send notification:
oracle.ons.Notification@17ca0f5
09/04/02 10:15:42.217 Send notification:
oracle.ons.Notification@1a28842
09/04/02 10:16:12.227 Send notification:
oracle.ons.Notification@5144d5
09/04/02 10:16:42.237 Send notification:
oracle.ons.Notification@19078ed
09/04/02 10:17:12.247 Send notification:
oracle.ons.Notification@fcc268
09/04/02 10:17:42.257 Send notification:
oracle.ons.Notification@16df388

A quick google turned up this page in which bug 7132128 (“OC4J EMITS “SEND NOTIFICATION” MESSAGES THAT FILL SERVER.LOG”) is identified.

The server.log is currently only 6MB and given the size of the server it will be a while before it causes us problems, but it’s worth being aware of.

A very minor irritation, but an irritation nonetheless, is when I go to Application Server Control in OAS I have to login twice.

Reading around I found this is an Apache feature, and is actually designed behaviour.

For reasons I’ve not explored our servers have several different hostnames which resolve to the same IP, e.g.:
myserver
myserver-app
myserver-data

When you request a page from Apache using a hostname other than that configured as ServerName in Apache’s httpd.conf, it redirects you to the version of the page using the ServerName.

(See here and here for history)

I edited the shell script which is eventually called by run-saw.sh to start the sawserver, (OracleBI)/setup/sawserver.sh, to use trus:
Comment out the final line:

$SASAWSERVER

and insert as a new line:

tusc -fepan -o /tmp/sawserver_tusc.out $SASAWSERVER

The output of trus ended with this:

open("/app/oracle/product/10.2.0/lib/libstd_v2.so.1", O_RDONLY|0x800, 0) ……… ERR#2 ENOENT
open("/app/oracle/product/obiee/server/Bin64/libstd_v2.so.1", O_RDONLY|0x800, 0) . ERR#2 ENOENT
open("/app/oracle/product/obiee/web/bin64/libstd_v2.so.1", O_RDONLY|0x800, 0) …. ERR#2 ENOENT
open("/app/oracle/product/obiee/odbc/lib64/libstd_v2.so.1", O_RDONLY|0x800, 0) … ERR#2 ENOENT
open("/usr/lib/libstd_v2.so.1", O_RDONLY|0x800, 0) …………………………. ERR#2 ENOENT
open("/lib/libstd_v2.so.1", O_RDONLY|0x800, 0) …………………………….. ERR#2 ENOENT
open("/app/oracle/product/10.2.0/lib/libstd_v2.so.1", O_RDONLY|0x800, 0) ……… ERR#2 ENOENT
open("/opt/aCC/lib/hpux64/libstd_v2.so.1", O_RDONLY|0x800, 0) ……………….. ERR#2 ENOENT

In investigating the problems with sawserver I was pointed towards a tool called tusc (which appears to be an HP version of truss).

You can use it to invoke a program, and get out a bunch of debug information including system calls.

You run it like this:

$tusc -fep /app/oracle/product/obiee/web/bin64/sawserver64

As a beginner when it comes to hardcore *nix I can only look at this and take pot shots at what’s going on, but with Google by my side I’m interested in the last lines of the output:

Follow on from setting up Publisher in a clustered environment, I’ve found a nasty little bug in the scheduling element of Publisher, Quartz.

Looking at the oc4j log file /opmn/logs/default_grouphomedefault_group~1.log I can see OC4J starting up, and then a whole load of repeated messages:

09/03/30 11:28:43 Oracle Containers for J2EE 10g (10.1.3.3.0) initialized
- ClusterManager: detected 1 failed or restarted instances.
- ClusterManager: Scanning for instance “myserver.fqdn.company.net1238408921404”’s failed in-progress jobs.
- ClusterManager: detected 1 failed or restarted instances.
- ClusterManager: Scanning for instance “myserver.fqdn.company.net1238408921404”’s failed in-progress jobs.
- ClusterManager: detected 1 failed or restarted instances.
- ClusterManager: Scanning for instance “myserver.fqdn.company.net1238408921404”’s failed in-progress jobs.
- ClusterManager: detected 1 failed or restarted instances.
- ClusterManager: Scanning for instance “myserver.fqdn.company.net1238408921404”’s failed in-progress jobs.
- ClusterManager: detected 1 failed or restarted instances.
- ClusterManager: Scanning for instance “myserver.fqdn.company.net1238408921404”’s failed in-progress jobs.
- ClusterManager: detected 1 failed or restarted instances.
- ClusterManager: Scanning for instance “myserver.fqdn.company.net1238408921404”’s failed in-progress jobs.
- ClusterManager: detected 1 failed or restarted instances.
- ClusterManager: Scanning for instance “myserver.fqdn.company.net1238408921404”’s failed in-progress jobs.
- ClusterManager: detected 1 failed or restarted instances.
- ClusterManager: Scanning for instance “myserver.fqdn.company.net1238408921404”’s failed in-progress jobs.
[… repeated for 38MB worth ]

We’re getting this error in the Presentation Services plug-in [analytics].
Log file: /j2ee/home/application-deployments/analytics/home_default_group_1/application.log

09/03/30 13:16:38.75 analytics: Servlet error
java.net.ConnectException: Connection refused (errno:239)
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333)
at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
at java.net.Socket.connect(Socket.java:517)
at java.net.Socket.connect(Socket.java:467)
at java.net.Socket.(Socket.java:364)
at java.net.Socket.(Socket.java:178)
at com.siebel.analytics.web.sawconnect.ConnectionPoolSocketFactoryImpl.createSocket(ConnectionPoolSocketFactoryImpl.java:63)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at com.siebel.analytics.web.sawconnect.ConnectionPoolSocketFactoryImpl.createSocket(ConnectionPoolSocketFactoryImpl.java:70)
at com.siebel.analytics.web.sawconnect.ConnectionPool.createNewConnection(ConnectionPool.java:314)
at com.siebel.analytics.web.sawconnect.ConnectionPool.getConnection(ConnectionPool.java:133)
at com.siebel.analytics.web.SAWBridge.processRequest(SAWBridge.java:299)
at com.siebel.analytics.web.SAWBridge.doGet(SAWBridge.java:325)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:743)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:856)
at com.evermind[Oracle Containers for J2EE 10g (10.1.3.3.0) ].server.http.ServletRequestDispatcher.invoke(ServletRequestDispatcher.java:713)
at com.evermind[Oracle Containers for J2EE 10g (10.1.3.3.0) ].server.http.ServletRequestDispatcher.forwardInternal(ServletRequestDispatcher.java:3
70)
at com.evermind[Oracle Containers for J2EE 10g (10.1.3.3.0) ].server.http.HttpRequestHandler.doProcessRequest(HttpRequestHandler.java:871)
at com.evermind[Oracle Containers for J2EE 10g (10.1.3.3.0) ].server.http.HttpRequestHandler.processRequest(HttpRequestHandler.java:453)
at com.evermind[Oracle Containers for J2EE 10g (10.1.3.3.0) ].server.http.AJPRequestHandler.run(AJPRequestHandler.java:302)
at com.evermind[Oracle Containers for J2EE 10g (10.1.3.3.0) ].server.http.AJPRequestHandler.run(AJPRequestHandler.java:190)
at oracle.oc4j.network.ServerSocketReadHandler$SafeRunnable.run(ServerSocketReadHandler.java:260)
at oracle.oc4j.network.ServerSocketAcceptHandler.procClientSocket(ServerSocketAcceptHandler.java:239)
at oracle.oc4j.network.ServerSocketAcceptHandler.access$700(ServerSocketAcceptHandler.java:34)
at oracle.oc4j.network.ServerSocketAcceptHandler$AcceptHandlerHorse.run(ServerSocketAcceptHandler.java:880)
at com.evermind[Oracle Containers for J2EE 10g (10.1.3.3.0) ].util.ReleasableResourcePooledExecutor$MyWorker.run(ReleasableResourcePooledExecutor.
java:303)
at java.lang.Thread.run(Thread.java:595)

I’m installing ODI agent on our database server using OUI. I selected the “Server” option at install time to get the Agent only, but looking in oracledi/bin odiparams.sh is missing:

$ls -l *.sh
-rwxrwxrwx 1 odiadm dba 685 Nov 21 15:58 agent.sh
-rwxrwxrwx 1 odiadm dba 908 Nov 21 15:58 agentscheduler.sh
-rwxrwxrwx 1 odiadm dba 707 Nov 21 15:58 agentstop.sh
-rwxrwxrwx 1 odiadm dba 941 Nov 21 15:58 agentweb.sh
-rwxrwxrwx 1 odiadm dba 724 Nov 21 15:58 jython.sh

If you work with a file in Windows and Unix at some point you might end up with windows line feed characters in your Unix file. It’ll look like this:

one line of text ^M
next line ^M
and next line with more ^M

To remove the ^M character, load the file into vi on unix and enter as a line command the following:

:1,$s/^M//

but instead of typing ^M do Ctrl-V Ctrl-M to get the charaters

We’re getting problems with an instance of Informatica / out-of-the-box OBIA on a new set of servers. When we run the execution plan we get this error soon after starting:

MAPPING> DBG_21075 Connecting to database [TNSENTRY], user [MYUSER]
MAPPING> CMN_1761 Timestamp Event: [Tue Mar 24 18:56:33 2009]
MAPPING> CMN_1022 Database driver error…
CMN_1022 [
Database driver error…
Function Name : Logon
ORA-12537: TNS:connection closed

Database driver error…
Function Name : Connect
Database Error: Failed to connect to database using user [MYUSER] and connection string [TNSENTRY].]

The Oracle BI Publisher Enterprise Cluster Deployment doc which I just found through Metalink highlighted a couple of points:
- Report repository should be shared
- The scheduler should be configured for a cluster

Report Repository
Through Admin>System Maintenance>Report Repository I changed the path from the default, /xmlp/XMLP to a NFS mount data/shared/xmlp and restarted the xmlpserver application in OAS. On coming back up Publisher complained because all its config files (in xmlp/Admin), had disappeared. I’d not moved any of the contents of /xmlp/XMLP since Report Repository suggested to me that it was just for reports, ergo with no reports yet created there was nothing to move.
So pedantaries aside, I moved the contents of /xmlp/XMLP to my new share, data/shared/xmlp. Publisher was happy after this.

• Delicious Bookmarks
• FireGestures
• FoxyProxy
• Greasemonkey
• HttpFox
• Screengrab
• User Agent Switcher

I’m learning about a lot of the Oracle BI stack through reading manuals and trial-and-error.
One thing I’ve realised is that Metalink holds a whole heap of useful information.
For example, simply searching on “publisher cluster” throws up these two very pertinent docs:

• OBIEE Clustered Installation with BI Publisher (Doc ID 744515.1)
• BI Publisher does not accept cluster jdbc connection strings (Doc ID 559795.1)

The first one is a publically available PDF, the second one is the answer to the problem I spent more time than I needed to scratching my head over yesterday.

In setting the scheduler in Publisher I discovered a useful difference in jdbc drivers.
Our repository is on Oracle 11g.
According to the manual oracle.jdbc.driver.OracleDriver should be used, but previous installations have used oracle.bi.jdbc.AnaJdbcDriver so I tried this too.

In experimenting with both I found you get more useful feedback from the second one. Here’s the same problem reported by both drivers:

· Exception [TOPLINK-4002] (Oracle TopLink - 11g Release 1 (11.1.1.0.0) (Build 080319)): oracle.toplink.exceptions.DatabaseException Internal Exception: java.sql.SQLException: ORA-28000: the account is locked Error Code: 28000

Following my previous work on configuring Publisher, I wanted to note down where the changes were written to.

The -mname syntax of the unix find command comes in handy here:

find /app/oracle/product/obiee -mtime -1

Shows me all files under the specified path which were modified in the last 1 day

and helpfully throws up:

/app/oracle/product/obiee/xmlp/XMLP/Admin/DataSource/datasources.xml