分类目录归档:系统层级

Oracle EBS weblogic 禁用筛选器T3导致weblogic服务不能重启问题

背景:
2020年1月15号安全网站出了一个 CVE-2020-2546,于是按照里面的“指示”,做了T3禁用,导致weblogic服务启动失败,EBS应用不能正常登陆,启动失败日常如下:

AdminServer logs are located at /u01/DEV/app/fs1/FMW_Home/user_projects/domains/EBS_domain_DEV/servers/AdminServer/logs

01/17/20-09:13:58 :: adadminsrvctl.sh: exiting with status 1

================================================================================

01/17/20-09:24:07 :: adadminsrvctl.sh version 120.10.12020000.10
Validated the passed arguments for the option ebs-get-serverstatus
AdminServer is currently not running.

Validated the passed arguments for the option ebs-nmstart-adminsrv
Checking if the Admin Server is already up.
The Admin Server is not already up.
FMW Version is 11.1.1.9
Checking if the Node Manager is already up..
Connecting to Node Manager …
Successfully Connected to Node Manager.

The Node Manager is already up.

Starting server AdminServer …
Error Starting server AdminServer: weblogic.nodemanager.NMException: Exception while starting server ‘AdminServer’

ERROR: Unable to connect the AdminServer.

StackTrace:
java.io.IOException
at weblogic.management.remote.common.ClientProviderBase.makeConnection(ClientProviderBase.java:209)
at weblogic.management.remote.common.ClientProviderBase.newJMXConnector(ClientProviderBase.java:97)
at javax.management.remote.JMXConnectorFactory.newJMXConnector(JMXConnectorFactory.java:369)
at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:267)
at oracle.apps.ad.util.WLUtil.initMBeanServerConnection(WLUtil.java:131)
at oracle.apps.ad.tools.configuration.EBSProvisioner.ebs_nmstart_adminsrv(EBSProvisioner.java:3682)
at oracle.apps.ad.tools.configuration.EBSProvisioner.ebs_nmstart_adminsrv(EBSProvisioner.java:3915)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at oracle.apps.ad.tools.configuration.EBSProvisioner.main(EBSProvisioner.java:8880)
Caused by: javax.naming.CommunicationException [Root exception is java.net.ConnectException: t3://devfin.guobaojinrong.com:7001: Destination unreachable; nested exception is:
java.net.ConnectException: 拒绝连接; No available router to destination]
at weblogic.jndi.internal.ExceptionTranslator.toNamingException(ExceptionTranslator.java:40)
at weblogic.jndi.WLInitialContextFactoryDelegate.toNamingException(WLInitialContextFactoryDelegate.java:792)
at weblogic.jndi.WLInitialContextFactoryDelegate.getInitialContext(WLInitialContextFactoryDelegate.java:366)
at weblogic.jndi.Environment.getContext(Environment.java:315)
at weblogic.jndi.Environment.getContext(Environment.java:285)
at weblogic.jndi.WLInitialContextFactory.getInitialContext(WLInitialContextFactory.java:117)
at javax.naming.spi.NamingManager.getInitialContext(NamingManager.java:684)
at javax.naming.InitialContext.getDefaultInitCtx(InitialContext.java:307)
at javax.naming.InitialContext.init(InitialContext.java:242)
at javax.naming.InitialContext.<init>(InitialContext.java:216)
at weblogic.management.remote.common.ClientProviderBase.makeConnection(ClientProviderBase.java:193)
… 11 more
Caused by: java.net.ConnectException: t3://devfin.guobaojinrong.com:7001: Destination unreachable; nested exception is:
java.net.ConnectException: 拒绝连接; No available router to destination
at weblogic.rjvm.RJVMFinder.findOrCreateInternal(RJVMFinder.java:216)
at weblogic.rjvm.RJVMFinder.findOrCreate(RJVMFinder.java:170)
at weblogic.rjvm.ServerURL.findOrCreateRJVM(ServerURL.java:165)
at weblogic.jndi.WLInitialContextFactoryDelegate$1.run(WLInitialContextFactoryDelegate.java:345)
at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:363)
at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:146)
at weblogic.jndi.WLInitialContextFactoryDelegate.getInitialContext(WLInitialContextFactoryDelegate.java:340)
… 19 more
Caused by: java.rmi.ConnectException: Destination unreachable; nested exception is:
java.net.ConnectException: 拒绝连接; No available router to destination
at weblogic.rjvm.ConnectionManager.bootstrap(ConnectionManager.java:470)
at weblogic.rjvm.ConnectionManager.bootstrap(ConnectionManager.java:321)
at weblogic.rjvm.RJVMManager.findOrCreateRemoteInternal(RJVMManager.java:262)
at weblogic.rjvm.RJVMManager.findOrCreate(RJVMManager.java:199)
at weblogic.rjvm.RJVMFinder.findOrCreateRemoteServer(RJVMFinder.java:238)
at weblogic.rjvm.RJVMFinder.findOrCreateInternal(RJVMFinder.java:200)
… 25 more

AdminServer logs are located at /u01/DEV/app/fs1/FMW_Home/user_projects/domains/EBS_domain_DEV/servers/AdminServer/logs

01/17/20-09:24:24 :: adadminsrvctl.sh: exiting with status 1

================================================================================

应用界面登陆也显示weblogic连接失败。

 

解决过程探索:

一般遇到这类问题,第一映像就是先恢复之前的设置,于是去找相关的表或者配置文件,从表中很难找到指向表,于是从另外一个正常的环境进入相应的weblogic主页面,并进入之前配置的页面查看帮助,于是找到了一个比较有指向性的文件(不知道具体文件在哪个目录下,通过find命令搜索),config.xml。

以下文件是来自帮助文件里面的内容:

Changes take effect after you redeploy the module or restart the server.

If this attribute configures a module that you deploy (such as an application or a JDBC data source that is part of an application) or a system resource whose configuration is saved in a descriptor file instead of in the domain’s config.xml file (such as a JDBC data source that is scoped at the system level), the module or resource cannot process the change until you redeploy it or restart its host server. If the module is a component in an application, Oracle recommends that you redeploy the entire application to avoid complications due to intra-application dependencies.

If this attribute configures some other part of the domain (such as a server, a cluster, or an EJB container), the system cannot process the change until you restart the server or cluster.

进入相应目录后,发现这个config.xml文件有很多序列号的文件名,如config39.xml,config38.xml,看这些文件的更新时间,推测应该是每次更新都会做一次备份,于是备份当前的config.xml文件,恢复最近的一个config.xml文件,然后重启weblogic服务,重启(正常重启,登陆页面也正常出现,并可登陆)如下文本:

[appldev@devfin scripts]$ sh adadminsrvctl.sh start

You are running adadminsrvctl.sh version 120.10.12020000.10

Enter the WebLogic Admin password:
Enter the APPS Schema password:
Starting WLS Admin Server…
Refer /u01/DEV/app/fs1/inst/apps/DEV_devfin/logs/appl/admin/log/adadminsrvctl.txt for details

AdminServer logs are located at /u01/DEV/app/fs1/FMW_Home/user_projects/domains/EBS_domain_DEV/servers/AdminServer/logs

adadminsrvctl.sh: exiting with status 0

adadminsrvctl.sh: check the logfile /u01/DEV/app/fs1/inst/apps/DEV_devfin/logs/appl/admin/log/adadminsrvctl.txt for more information …

说明:ebs weblogic尽量少变动,在改动前也一定要做好备份。

相关参考文件:
https://www.oracle.com/security-alerts/cpujan2020.html

https://docs.oracle.com/cd/E23943_01/web.1111/e13707/ssl.htm#SECMG389

http://www.ijiandao.com/2b/baijia/345629.html

https://support.oracle.com/epmos/faces/DocumentDisplay?_afrLoop=508979212187743&id=1280374.1&_afrWindowMode=0&_adf.ctrl-state=e2y1sye3v_1027

Oracle EBS R12.1.3登陆环境问题(JSP问题)

错误描述:

Unable to generate forwarding URL. Exception: oracle.apps.fnd.common.AppsException: java.lang.NullPointerException: null Connection

方案参考:
EBS Homepage Errors Unable To Generate Forwarding URL Exception oracle.apps.fnd.common.AppsException java.lang.NullPointerException null Connection (文档 ID 2055421.1)

说明:查看文档,发现官方是认为系统缓存或者某些jsp文件crash造成,因此主要思路是将_pages相关jsp文件进行整体重编译,然后删除相关cache文件来解决。

执行过程(关闭应用及备份删除相关操作、重启应用不在此记录):

[appldev@erpdevap bin]$ ./ojspCompile.pl –compile –flush -p 2
logfile set: /app/DEV/inst/apps/DEV_erpdevap/logs/appl/rgf/ojsp/ojspc_error.log
starting…(compiling all)
using 10i internal ojsp ver: 10
synchronizing dependency file:
enumerating jsps…8095
parsing jsp…8095
writing deplist…8095
initializing compilation:
eliminating children…6024 (-2071)
translating and compiling:
translating jsps…6024/6024 in 1m33s
compiling jsps…6024/6024 in 6m20s
Finished!

跟踪错误日志:

[BEGIN] 2020/1/13 17:38:11
[appldev@erpdevap _pages]$ tailf /app/DEV/inst/apps/DEV_erpdevap/logs/appl/rgf/ojsp/ojspc_error.log
[19659] compiling: 7s elapsed, 50 successful 0 failed
[19658] compiling: 7s elapsed, 50 successful 0 failed
[19659] compiling: 6s elapsed, 50 successful 0 failed
[19659] compiling: 4s elapsed, 50 successful 0 failed
[19658] compiling: 7s elapsed, 50 successful 0 failed
[19659] compiling: 2s elapsed, 50 successful 0 failed
[19658] compiling: 3s elapsed, 50 successful 0 failed
[19659] compiling: 2s elapsed, 24 successful 0 failed
COMPILED: 6024 [failed: 0] in 6m20s
18890 FINISHING Mon Jan 13 17:36:33 2020

[appldev@erpdevap _pages]$ exit
logout

[END] 2020/1/13 17:40:35

weblogic server psu

参考文件:Critical Patch Update (CPU) Program July 2019 Patch Availability Document (PAD) (文档 ID 2534806.1)

对Oracle EBS(R12.2)系统而言,由于已经集成weblogic服务,特别是双文件系统(RUN、PATCH),只需要对RUN系统进行最新补丁(后面做adop cycle的时候回自动同步,即做patch时切换模式会自动同步)

以下命令是查询系统补丁情况,如果打上了补丁,则会有显示:

cd $FMW_HOME/utils/bsu

./bsu.sh -view -verbose -status=applied -prod_dir=/app/test/apps/fs1/FMW_Home/wlserver_10.3/

 

Content:
========
This patch contains Smart Update patch 8K1U for WebLogic Server 10.3.6.0

Description:
============
WEBLOGIC SAMPLES SPU 10.3.6.0.190716

Patch Installation Instructions:
================================
- copy content of this zip file with the exception of README file to your SmartUpdate cache directory (FMW_HOME/utils/bsu/cache_dir by default)
- apply patch using Smart Update utility

Oracle增加表空间步骤

本文只介绍新增文件的方式(其他方式类似可自行研究):

1、通过命令:show parameter db_block_size,来查询当前数据库单个文件的最大size,如果是8192(8K),最大单个文件为32G,如果是16384(16K),最大单个文件为64G。

2、通过以下命令查询当前系统对应需要增加数据文件的表空间信息。

SELECT Upper(F.TABLESPACE_NAME) “表空间名”,
D.TOT_GROOTTE_MB “表空间大小(M)”,
D.TOT_GROOTTE_MB – F.TOTAL_BYTES “已使用空间(M)”,
To_char(Round((D.TOT_GROOTTE_MB – F.TOTAL_BYTES) / D.TOT_GROOTTE_MB * 100,
2),
‘990.99’) || ‘%’ “使用比”,
F.TOTAL_BYTES “空闲空间(M)”,
F.MAX_BYTES “最大块(M)”
FROM (SELECT TABLESPACE_NAME,
Round(Sum(BYTES) / (1024 * 1024), 2) TOTAL_BYTES,
Round(Max(BYTES) / (1024 * 1024), 2) MAX_BYTES
FROM SYS.DBA_FREE_SPACE
GROUP BY TABLESPACE_NAME) F,
(SELECT DD.TABLESPACE_NAME,
Round(Sum(DD.BYTES) / (1024 * 1024), 2) TOT_GROOTTE_MB
FROM SYS.DBA_DATA_FILES DD
GROUP BY DD.TABLESPACE_NAME) D
WHERE D.TABLESPACE_NAME = F.TABLESPACE_NAME
ORDER BY 1;

3、通过以下命令查询表空间对应单个文件的信息(主要是保证新增的数据文件跟原有文件保存信息一致,如保证单个文件一致)。

SELECT “File Name”, “Tablespace”, “Status”, “Size (MB)”, “Used (MB)”, “Used (Proportion)”, “Used (%)”, “Auto Extend” FROM(
select * from (
SELECT /*+ all_rows use_concat */
‘SQLDEV:LINK{#;#}’||USER||’#;#DATAFILE#;#’||ddf.file_name||’#;#oracle.dbtools.raptor.dba.navigator.Drill.DBADrillLink’ as “File Name”,
ddf.tablespace_name as “Tablespace”,
ddf.online_status as “Status”,
TO_CHAR(NVL(ddf.bytes / 1024 / 1024, 0), ‘99999990.000’) as “Size (MB)”,
TO_CHAR(DECODE(NVL(u.bytes/1024/1024, 0), 0, NVL((ddf.bytes – NVL(s.bytes, 0))/1024/1024, 0), NVL(u.bytes/1024/1024, 0)), ‘99999999.999’) as “Used (MB)”,
CASE
when ddf.online_status = ‘OFFLINE’ then
‘OFFLINE’
when ddf.online_status = ‘RECOVER’ then
‘RECOVER’
else
‘SQLDEV:GAUGE:0:100:0:0:’|| TRIM(TO_CHAR(DECODE((NVL(u.bytes, 0) / ddf.bytes * 100), 0, NVL((ddf.bytes – NVL(s.bytes, 0)) / ddf.bytes * 100, 0), (NVL(u.bytes, 0) / ddf.bytes * 100)), ‘990’))
end as “Used (Proportion)”,
TO_CHAR(DECODE((NVL(u.bytes, 0) / ddf.bytes * 100), 0, NVL((ddf.bytes – NVL(s.bytes, 0)) / ddf.bytes * 100, 0), (NVL(u.bytes, 0) / ddf.bytes * 100)), ‘990.99’) as “Used (%)”,
ddf.autoextensible as “Auto Extend”
FROM
sys.dba_data_files ddf,
(
SELECT
file_id,
SUM(bytes) bytes
FROM
sys.dba_free_space GROUP BY file_id
) s,
(
SELECT
file_id,
SUM(bytes) bytes
FROM
sys.dba_undo_extents
WHERE
status <> ‘EXPIRED’
GROUP BY file_id
) u
WHERE
(ddf.file_id = s.file_id(+) and ddf.file_id=u.file_id(+))
UNION
SELECT
‘SQLDEV:LINK{#;#}’||USER||’#;#DATAFILE#;#’||v.name||’#;#oracle.dbtools.raptor.dba.navigator.Drill.DBADrillLink’ as “File Name”,
dtf.tablespace_name as “Tablespace”,
dtf.status as “Status”,
TO_CHAR(NVL(dtf.bytes / 1024 / 1024, 0), ‘99999990.000’) as “Size (MB)”,
TO_CHAR(NVL(t.bytes_used/1024/1024, 0), ‘99999990.000’) as “Used (MB)”,
CASE
when dtf.status = ‘OFFLINE’ then
‘OFFLINE’
else
‘SQLDEV:GAUGE:0:100:0:0:’|| TRIM(TO_CHAR(NVL(t.bytes_used / dtf.bytes * 100, 0), ‘990.99’))
end as “Used (Proportion)”,
TO_CHAR(NVL(t.bytes_used / dtf.bytes * 100, 0), ‘990’) as “Used (%)”,
dtf.autoextensible as “Auto Extend”
FROM
sys.dba_temp_files dtf,
sys.v_$tempfile v,
v$temp_extent_pool t
WHERE
(dtf.file_name = v.name or dtf.file_id = v.file#)
and dtf.file_id = t.file_id(+)
ORDER BY 1
) sub1 order by 2 asc
)

4、新增文件命令。

alter tablespace APPS_TS_TX_DATA add datafile ‘/u01/DEV/db/data/a_txn_data07.dbf’ size 10000m;

ORA-20100 文件 通过 FND_FILE 创建失败解决方案

克隆了一个EBS 11i的环境,提交客户同步请求的时候出现了错误提示是:
原因:由于 ORA-20100: 为 FND_FILE 创建文件 10034190.tmp 失败。
我的服务器中有2个ERP环境,参数APPSLPTMP指定为默认的/usr/tmp。然后, 在网上找了一些相关资料,说不同的环境APPSLPTMP和数据库的utl_file_dir参数不能指定相同的目录,于是做了修改,修改命令如下:
# mkdir -p /usr/opt/tmp       创建目录
# chmod -R a+rw /usr/opt  赋予该目录读写权限
# su – orauat                               切换致数据库用户
$ export APPSLPTMP=/usr/opt/tmp   修改环境变量
$ cd $ORACLE_HOME/dbs
$ vi init<SID>.ora                   编辑数据库初始化文件,修改参数:utl_file_dir
utl_file_dir = /usr/opt/tmp,……..<SID>_prod
并且重新启动的环境。在数据库执行函数:FND_FILE_OUT_LINE()提示成功。
这个错误很常见:

1.查看$APPLPTMP系统环境变量的值,一般是/usr/tmp,需要保证该文件夹是存在的;

2.查看utl_file_dir数据库参数,其第一个值也应该为/usr/tmp;
select* from v$parameter t whee t.name=’utl_file_dir’
3.查看该文件夹的权限,该文件夹必须为应用用户和数据库用户都具有读写权限;
4.通过exec FND_FILE.PUT_LINE(FND_FILE.LOG, ‘THIS IS A TEST’);进行测试;
5.如果仍然还有问题,请查看你的服务器上面是不是有多套ERP环境,如果有多个的话两个$APPLPTMP文件同时写会冲突当一台服务器上运行了多套环境时,不能使用/usr/tmp作为$APPLPTMP,须定义成各自的目录。且该目录须在数据库参数utl_file_dir中。