oracle.security.jps.wls.listeners.JpsApplicationLifecycleListener Class not Found

Resolving the Exception

"oracle.security.jps.wls.listeners.JpsApplicationLifecycleListener Class not Found"  

  On re-start of a new install of OID11g on weblogic, the following error was seen:

'oracle.security.jps.wls.listeners.JpsApplicationLifecycleListener' Class not Found 

This strongly indicates an issue with the exception in Weblogic environment. Also the deployments show remain in 'Prepared' state.

The error can be fixed with setting the nodemanager property to use the weblogic startup script by changing the parameter "StartScriptEnabled=true"

This will include correct the environment during startup.

The deployments will also be activated effectively on startup.

Problems with reinstalling SOA [ERROR]:INST-07290

OIM OAM SOA implementation

I had to install OIM OAM and SOA a few times before I got it right. The box also had OID implemented on it with several thousand users. I could not afford to purge the OID datastore every time I had to reinstalled the OIM. Down the way, I had trouble with installing more than one installations of SOA in the same middleware home. 

The error stated as below :

[VALIDATION] [ERROR]:INST-07290: Specified Middleware home already has a Oracle SOA Suite 11g home. Only one Oracle SOA Suite 11g home location can reside under a Middleware home.

[VALIDATION] [SUGGESTION]:Provide a different Middleware home location

installation Failed. Exiting installation due to data validation failure.

 I found an easy workaround to get past the problem.

I had to backup the inventory.xml file in the global inventory location and comment the REFHOME LOC lines that point to the corrupted install as seen in the screenshot below :

 Then after purging the domain applications and after correcting the file $MW_HOME/domain-registry.xml, I would be all set to reinstall OIM OAM and SOA allover once again.

OID11g DIP migrating ldap-server to a new-host

Reconfiguring AD (Domain controller)

AD – OID synchronization is setup as explained in the document  : http://docs.oracle.com/cd/E14571_01/admin.1111/e10031/odip_adm_sync.htm#OIMIG1934

For any reconfiguration relating to AD such as demoting/renaming etc, the configuration has to be updated in the following procedure :  

• Identify the “HighestCommittedUSN” of the  domain controller in use by the enabled synchronization profile. (different replicating domain controllers )

 ldapsearch –h –p 389 –D –w “ –b “” –s base “objectclass=*” highestCommittedUSN

highestCommittedUSN=3537140

•     determine the “HighestCommittedUSN” of the  domain controller to which it should be reconfigured to

ldapsearch –h –p 389 –D –w “ –b “” –s base “objectclass=*” highestCommittedUSN

 highestCommittedUSN=31243468

• Disable the current synchronization profile

•    Edit the synchronization profile to connect to the appropriate domain controller ( in this example )

•     Reset the last applied change number

- Go to the Advanced Tab

- In the section "Additional Configuration Parameters" edit 'Skip Error To Sync Next Change' and set to true

- Check "Edit and Persist" box, then Change "Last Change Number" value to a value less than highestCommittedUSN of the new DC (in this case 31243468)

Save the change

•   Enable the synchronization profile
• Monitor synchronization completes without errors

NFS nolock option

Maximum availability architecture recommends clustered deployment of middleware components and for the admin server to be positioned on a NFS mount which can switch between the clustered nodes.

I implemented one such architecture and encountered the NFS Nolock problem recently. Here is the error and a safe resolution.

======
While attmpting to startup nodemanager using wlst, the node manager starts up fine. however, the folowing error is seen. if this error is ignored, we could landup wasting a lot of time troubleshoting further.

Jul 16, 2015 11:34:35 PM weblogic.nodemanager.server.NMServer initDomains
WARNING: Configuration error while reading domain directory: /mnt/oracle/****/****/**_Domain
java.io.IOException: No locks available
     at sun.nio.ch.FileChannelImpl.lock0(Native Method)
     at sun.nio.ch.FileChannelImpl.lock(FileChannelImpl.java:832)
     at weblogic.nodemanager.server.UserInfo.load(UserInfo.java:149)
     at weblogic.nodemanager.server.DomainManager.loadUserInfo(DomainManager.java:139)
     at weblogic.nodemanager.server.DomainManager.initialize(DomainManager.java:90)
     at weblogic.nodemanager.server.DomainManager.(DomainManager.java:60)
     at weblogic.nodemanager.server.NMServer.initDomains(NMServer.java:225)
     at weblogic.nodemanager.server.NMServer.start(NMServer.java:197)
     at weblogic.nodemanager.server.NMServer.main(NMServer.java:382)
     at weblogic.NodeManager.main(NodeManager.java:31)

Jul 16, 2015 11:34:35 PM weblogic.nodemanager.server.Listener run
============

While starting Admin server, using wlst, the following error will be seen  :

There are 1 nested errors:
weblogic.management.ManagementException: Unable to obtain lock on /user_projects/domains/biadm10_bifoundation_domain/servers/AdminServer/tmp/AdminServer.lok. Server may already be running
 
 =============
90% of the problem is resolved when you have determined that the problem is due to NFS and not due to the non availability or availability of the lck file :) and the remaining 10% is to make a small edit on  /etc/fstab

Note (Doc ID 1635553.1) has the solution explained. I had the problem resolved by enable the nolock mount option.

nolock,vers=3,tcp,timeo=300
nolock,vers=3,tcp,timeo=300


Cheers :)
Arun
 

FRM 93552

FRM 93552

I had to administer a new implementation n 11.1.2 version of forms and reports server.

I had a good chance to learn that there are a few fundamental differences in 10g and 11g version of forms.
It is best practice in 11g to make formweb.cfg and other config file changes by using the em console.  sometime it becomes an intermittent problem in case this thumb-rule is ignored.

Following forms error is a manifestation of a manual change made to formsweb.cfg on the forms server.

 

 In case the error is accompanied with the following errors, as is seen in the diagnostics log here :

 

There is a oracle support note that recommends a native library :

Getting FRM-93552 When Calling Forms (Doc ID 1345748.1)

here the recommendation is to include the actual path for ORACLE_HOME/jdk/jre/lib/amd64/native_threads LD_LIBRARY_PATH variable in the ******.env file. Again, it is best to update the *****.env file over the em console

 

This was a good lesson learned!!

Cheers, 

Arun

Volume check

In one project, We had a service provider supporting storage and infrastructure.

One fine morning we heard there would be a network outage impacting all services. This meant an overnight schedule involving :

• Prep work
• Prevention of autostart scripts/crons/batches and the enterprise scheduler
• Shutdown of the whole production stack prior to the outage
• Manual startup
• Validation of services
• Enable autostart, cron, batches and enterprise scheduler
• makeup lost schedule executions 
• Wrap up

This was a tedious schedule spanning several servers, several DBAs working through the night.

The service provider added that the storage may be drawn away from the servers during the outage and nfs volumes need to be validated before startup.

I put together a handy script to put to rest the possibility of manual errors in the wee hours of the morning
following is the script attached  :) --> enjoy

nfs volume check

Datacenter move and IP change

From another datacenter move project,

Several weblogic middleware servers were being moved from one datacenter to another. Weblogic is not touchy with IPs however, it stores hostnames at several different locations Oracle told us on one of the SRs we had open with them. Soon I realized that the Weblogic config files also store in them at several locations references to IPs. 

There were several weblogic servers being moved and time was of the essence in this situation. I had to design a quick and simple script to correct IPs in the config files to have the weblogic servers to start smooth after the move.

Please find the script attached from my repository attached here -->
IP Change script

Datacenter move and system check

I was working on a datacenter move project recently,

We had to create tons of servers needed to be created and infrastructure was to be moved from old outdated hardware to newer hardware at the new datacenter. I had the following procedure advised to the sysadmin team for preparing the servers.

• RDA was run with the HCVE option for middleware, database and EBS specifications.
• Corrections were accordingly implemented to nominated servers
• Templates were to be created from the corrected servers.
• All new servers were to be created from the templates that were created thus.

The images created hence were noticed to be a little different from the other one. These were SA configurable parameters. Thereby, I had to come up with a script that would check and identify miss-configurations that would come after the sysadmins. The sysadmins were then advised to run the script and hand over the server to the DBA team only after all errors were corrected.

Please find the script attached from my repository attached here -->
System check