ORA-00904: "TABLE_FAMILY_ID": invalid identifier

 Problem:

--------------

Service does not start

oramkk@dogan:/home/oracle$ srvctl start service -s NECOSRV -d NECODB    

PRCD-1084 : Failed to start service NECOSRV

PRCR-1079 : Failed to start resource ora.necodb.necosrv.svc

CRS-5017: The resource action "ora.necodb.necosrv.svc start" encountered the following error: 

ORA-00904: "TABLE_FAMILY_ID": invalid identifier

. For details refer to "(:CLSN00107:)" in "/u01/app/oracle/diag/crs/dogan/crs/trace/ohasd_oraagent_oracle.trc".

From the trace I see that srvctl couldn't create service,but add service command completed successfully.I thing the problem is our grid version 19.3 but database version is 12.2.

Solution:

-------------

Create service and start from the database.

SQL> exec dbms_service.create_service('NECOSRV','NECOSRV');

PL/SQL procedure successfully completed.

oracle@dogan:/home/oramkk$ srvctl status service -s NECOSRV -d NECODB           

Service NECOSRV is not running.

SQL> BEGIN

  2    DBMS_SERVICE.start_service(

  3      service_name => 'NECOSRV'

  4    );

  5  END;

  6  /

PL/SQL procedure successfully completed.

oracle@dogan:/home/oracle$ srvctl status service -s NECOSRV -d NECODB

Service NECOSRV is running.

Mongodb Change secondary priority(avoid becoming primary)

Hi,

You may have mongodb replica set and also you may want to set ones to avoid becoming primary.

For example we have 1 replica set. 1 primary(Roma),1 local secondary(Roma) and 1 remote secondary(Milano). If there is a problem with primary Roma or Milano may be primary after elections.But we want that Milano site is disaster site and it shouldn't be in elections.

For this request,our need to set priority to 0



1)cfg = rs.conf()

2)cfg.members[2].priority = 0

3) rs.reconfig(cfg)



Attention:

cfg.members[2].priority------------------>We set 2nd member priority,it may be different in your replica set


MongoDB is pretty:)

Failed: necodb.contents: error creating collection necodb.contents: error running create command: BSON field 'OperationSessionInfo.create' is a duplicate field

Hi,

I have a migration project.We will migrate from mongo 2.4.12 to mongo 4.0.6.

We are using community version. So  we have to choose mongodump method.

There is no error while exporting data.But import is failed.


Problem:
-----------

##############################################################################
Failed: necodb.contents: error creating collection necodb.contents: error running create command: BSON field 'OperationSessionInfo.create' is a duplicate field


##############################################################################

Solution:
-----------

1)use 2.4.12 mongobackup on the 2.4 database

mongodump --db necodb--out /mondirec/backup/

2)delete or move all .json files from the outputted dump directory

3)use 4.0.6 mongorestore on the remaining .bson files

mongorestore --db necodb --drop /mondirec/backup/necodb




Thanks for dba.exchange.com for this solution.


Reference:

https://dba.stackexchange.com/questions/201827/failed-to-import-a-mongodb-database-with-duplicate-fields

Exadata Disk Scrubbing Problem

Problem:
--------

We hit a problem previous weeks related Exadata storage. In fact not related to Exadata but the impact is big.

We are using Attunity Replicate for a CDC tool and Attunity Replicate CDC method is different from goldengate.

As you know Goldengate uses the stream for reading redo logs and archives. But Attunity Replicate is different.

(I will use shortcut AR for Attunity replicate)

AR is reading online redo logs and archive logs with Oracle DBMS package so it may be affected by disk IO or anything else.


Solution:
-----------
Another story that;

Disk scrubbing is a new feature that introduced in Oracle 11.2.0.4 and Exadata 11.2.3.3.0 storage.

Disk scrubbing checks the disks for the corruption risks.

In default configuration,checks every 2 weeks.(hardDiskScrubInterval=biweekly)

Let me show;

It is the default configuration

CellCLI> list cell attributes name,hardDiskScrubInterval
         x6celadm05   biweekly

Till this step, everything is ok, Oracle protects our disks and data. But every cells start scrubbing same time, you may have IO problem.

Our database has a lot of transactions, but we don't see any impact of this scrubbing, till AR went slow done(latency is growing) while scrubbing is continues.

After we found the scrubbing information from cell alert logs, we decide to start scrubbing different times. Then we set a different start time
for every cell.

For example;

One cell disk scrubbing takes 44 hours and we have 7 cells. So I didn't change the interval(biweekly). After Cell 1 disk scrubbing is finished,
cell 2 will start, after cell 2 is finished cell 3 will start etc. So in this method, only one cell will do scrubbing in a meantime.

dcli -g cell_group  -l root cellcli -e "list cell attributes name,hardDiskScrubInterval,hardDiskScrubStartTime"
x6celadm07: x6celadm07     biweekly        2018-04-22T05:00:00+03:00
x6celadm06: x6celadm06     biweekly        2018-04-20T08:00:00+03:00
x6celadm05: x6celadm05     biweekly        2018-04-18T11:00:00+03:00
x6celadm04: x6celadm04     biweekly        2018-04-16T14:00:00+03:00
x6celadm03: x6celadm03     biweekly        2018-04-14T17:00:00+03:00
x6celadm02: x6celadm02     biweekly        2018-04-12T20:00:00+03:00
x6celadm01: x6celadm01     biweekly        2018-04-10T23:00:00+03:00

After this implementation, AR(attunnity replicate) latencies was going down.

ORA-00979: not a GROUP BY expression

Problem:

-----------

ORA-00979: not a GROUP BY expression

We have upgraded our database from 11.2.0.4 to 12.2.0.1.

After upgrade one sql got "ORA-00979: not a GROUP BY expression" error.

Solution:

-----------

It was similar to Bug 18749211 ORA-979 FROM SELECT WITH COLUMN MASKING VPD AND VIEW MERGING VPD, but we don't use VPN or anything else, only we upgraded the database.

Also, /*+ materialize */ didn't work advice as a workaround in bug document.

Then I found another bug 27170305: ORA-00979 WHEN A CASE STATEMENT IS THERE IN THE "GROUP BY" EXPRESSION 

Workaround:

---------------------

/*+ optimizer_features_enable('11.1.0.6') */

But these hints caused performance problems, we need to apply the patch if it could be done :)

So, if this patch is not available for your version like us, you need to ask to oracle support.

Now, this patch is ready for our system in a one week:)

Netezza Backup & Restore via Networker

EMC Networker supports Netezza backup configuration.Also we use this backup methology.

Previous week we deleted some row unfortunately.So we had to restore this table before the deletion was completed.

But we are newbie this issue,netezza restore.

Finally we hit restore command and restored it.But default restore without any date parameter,networker restores last backup.But I need to restore 3 previous backup.

So we hit the 3 previous backup set id from the backup log.

Then we restored it.

Backup command:
-----------------------
nzbackup -db XX_DWH -connector networker -connectorArgs NSR_SERVER=networkerist.doganay.com.tr:NSR_DATA_VOLUME_POOL=DDBoostNetezza


Restore Command:
------------------------
nzrestore -db XX_DWH_RST -sourcedb XX_DWH -backupset 20151017201912 -connector networker -connectorArgs NSR_SERVER=networkerist.doganay.com.tr:NSR_DATA_VOLUME_POOL=DDBoostNetezza -tables  XX_HIST -sourceschema XX_ADMIN


XX_DWH_RST -------->Empty database which created by us.

XX_DWH --------------->Source database which backed up.

20151017201912------->Backup set id(You can't find this id from backup admin,you have to mine netezza backup log)

networkerist.doganay.com.tr-------->networker hostname

XX_HIST --------------->table name which you want to restore it.

XX_ADMIN------------>table's schema



Restore output:
-------------------
Restore of increment 1 from backupset 20151017201912 to database 'XX_DWH_RST' committed.
Restore of increment 2 from backupset 20151017201912 to database 'XX_DWH_RST' committed.
Restore of increment 3 from backupset 20151017201912 to database 'XX_DWH_RST' committed.
Restore of increment 4 from backupset 20151017201912 to database 'XX_DWH_RST' committed.
Restore of increment 5 from backupset 20151017201912 to database 'XX_DWH_RST' committed.
Restore of increment 6 from backupset 20151017201912 to database 'XX_DWH_RST' committed.


Pray & Tray:)

ORA-01804 failure to initialize timezone information dbua

Problem:
-----------


ORA-01804 "failure to initialize timezone information"

While upgrading 11.1.0.7 database to 12.1.0.2,I got an error when run the dbua.


Solution:
-----------


unset ORA_TZFILE

Crfclust.bdb IS TOO BIG

Problem:
------------

Crfclust.bdb file is growing.

We have ODA machine which is RAC with two nodes.

Everyday we have got /u01 disk space alert.We cleaned trace files,cdmp,incdir vs.But disk space a bit free.

One day I found crfclust.bdb file which size is 32G in $GRID_HOME

This file used to Cluster Health Monitor.But there is a problem with its size.

This is a bug which related to ODA RAC system.

[NECO1]/u01/app/11.2.0.4/grid/crf/db/neco1 $ du -sh crfclust.bdb
32G     crfclust.bdb



Solution:
----------

[grid@neco1 ~]$ oclumon manage -get repsize

CHM Repository Size = 204737600

 Done

 This size is repositery retention.Our retention is 6,58 year.This is incredible:))

 This size should be between 3600 (1 hour) and 259200 (3 days).

------------------------------------------------------------------------------------
[grid@neco2 ~]$ oclumon manage -repos resize 259200
neco1 --> retention check successful
neco2 --> retention check successful
New retention is 259200 and will use 4524595200 bytes of disk space
CRS-9115-Cluster Health Monitor repository size change completed on all nodes.

 Done
------------------------------------------------------------------------------------

4524595200 bytes means 4GB.
------------------------------------------------------------------------------------

Then check the size;



[grid@neco2 ~]$ oclumon manage -get repsize

CHM Repository Size = 259200

 Done
--------------------------------------------------------------------------------------------------------------------------
If you get error like below while running 'oclumon manage -get repsize',you should stop and start ora.crf via 'crsctl stop res ora.crf -init'

CRS-9011-Error manage: Failed to initialize connection to the Cluster Logger Service
--------------------------------------------------------------------------------------------------------------------------


In my case I didn't need to bounce the ora.crf.


It is very nice size:))

[root@neco1 neco1]# du -sh crfclust.bdb
2.2M    crfclust.bdb



Reference note:
-------------------
ODA Nodes Lacking Space Due to Large Cluster Health Monitor File Crfclust.Bdb (Doc ID 1616910.1)



 

RMAN-06054: media recovery requesting unknown archived log for thread 2 with sequence and starting SCN of

PROBLEM:
---------------

RMAN-06054: media recovery requesting unknown archived log for thread 2 with sequence 8290 and starting SCN of 6062404529202



SOLUTION:
----------------

select scn_to_timestamp(6062404529202) from dual

output:
----------
01.07.2015 18:27:56,000000000

Also my backup finished at 18:34.But I couldn't recover at this point.

I found the 18:27 scn,then I set it.

select TIMESTAMP_TO_SCN (TO_TIMESTAMP ('20150701 18:27:00', 'YYYYMMDD HH24:MI:SS')) scn FROM dual;

output:
---------
6062404514529

Because redo didn't archived before the backup finished.So,this archive generated after finished backup.



RUN
{
allocate channel t1 type disk ;
allocate channel t2 type disk ;
allocate channel t3 type disk ;
allocate channel t4 type disk ;
set until scn 6062404514529;
SET NEWNAME FOR DATAFILE 1 to '+DATAC1';
SET NEWNAME FOR DATAFILE 15 to '+DATAC1';
SET NEWNAME FOR DATAFILE 2 to '+DATAC1';
SET NEWNAME FOR DATAFILE 4 to '+DATAC1';
SET NEWNAME FOR DATAFILE 23 to '+DATAC1';
SET NEWNAME FOR DATAFILE 24 to '+DATAC1';
SET NEWNAME FOR DATAFILE 41 to '+DATAC1';
SET NEWNAME FOR DATAFILE 42 to '+DATAC1';
SET NEWNAME FOR DATAFILE 5 to '+DATAC1';
SET NEWNAME FOR DATAFILE 6 to '+DATAC1';
SET NEWNAME FOR DATAFILE 26 to '+DATAC1';
SET NEWNAME FOR DATAFILE 27 to '+DATAC1';
SET NEWNAME FOR DATAFILE 28 to '+DATAC1';
SET NEWNAME FOR DATAFILE 29 to '+DATAC1';
SET NEWNAME FOR DATAFILE 30 to '+DATAC1';
SET NEWNAME FOR DATAFILE 31 to '+DATAC1';
SET NEWNAME FOR DATAFILE 32 to '+DATAC1';
SET NEWNAME FOR DATAFILE 33 to '+DATAC1';
SET NEWNAME FOR DATAFILE 34 to '+DATAC1';
SET NEWNAME FOR DATAFILE 37 to '+DATAC1';
SET NEWNAME FOR DATAFILE 38 to '+DATAC1';
SET NEWNAME FOR DATAFILE 39 to '+DATAC1';
SET NEWNAME FOR DATAFILE 40 to '+DATAC1';
SET NEWNAME FOR DATAFILE 7 to '+DATAC1';
SET NEWNAME FOR DATAFILE 8 to '+DATAC1';
SET NEWNAME FOR DATAFILE 9 to '+DATAC1';
SET NEWNAME FOR DATAFILE 10 to '+DATAC1';
SET NEWNAME FOR DATAFILE 11 to '+DATAC1';
SET NEWNAME FOR DATAFILE 12 to '+DATAC1';
SET NEWNAME FOR DATAFILE 13 to '+DATAC1';
SET NEWNAME FOR DATAFILE 14 to '+DATAC1';
SET NEWNAME FOR DATAFILE 17 to '+DATAC1';
SET NEWNAME FOR DATAFILE 18 to '+DATAC1';
SET NEWNAME FOR DATAFILE 19 to '+DATAC1';
SET NEWNAME FOR DATAFILE 20 to '+DATAC1';
SET NEWNAME FOR DATAFILE 22 to '+DATAC1';
SET NEWNAME FOR DATAFILE 60 to '+DATAC1';
SET NEWNAME FOR DATAFILE 25 to '+DATAC1';
SET NEWNAME FOR DATAFILE 3 to '+DATAC1';
SET NEWNAME FOR DATAFILE 16 to '+DATAC1';
SET NEWNAME FOR DATAFILE 21 to '+DATAC1';
restore database;
recover database;
release channel t1;
release channel t2;
release channel t3;
release channel t4;
}

Finished restore at 03-07-2015 12:05:29

Starting recover at 03-07-2015 12:05:29

starting media recovery
media recovery complete, elapsed time: 00:00:07

Finished recover at 03-07-2015 12:05:37




RMAN> alter database open resetlogs;

database opened




Pray and tray:)

OGG-02091 Operation not supported because enable_goldengate_replication is not set to true.

PROBLEM:
---------------

After you have installed you goldengate on 12C database,while starting replicat process replicat process may abended.

In my case source database is 11G,destination database is 12C.

After hit this error,I examined the ggserr.log is below:

[oracle@ahebsfin1 ggs_core]$ tail -100f ggserr.log

#################################################################################
2015-06-09 04:07:38  ERROR   OGG-02091  Oracle GoldenGate Delivery for Oracle, rep1.prm:  Operation not supported because enable_goldengate_replication is not set to true.

#################################################################################


SOLUTION:
----------------


enable_goldengate_replication is new parameter in 12C.

You have to enable this parameter for starting replicat process.


SQL> sho parameter goldengate

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
enable_goldengate_replication        boolean     FALSE


We set this parameter to TRUE.

SQL> alter system set enable_goldengate_replication=TRUE;


System altered

This is dynamic parameter.Set to TRUE,then replicat can start wihout any problem.



GGSCI (ahebsfin1) 1> info all

Program     Status      Group       Lag at Chkpt  Time Since Chkpt

MANAGER     RUNNING                                          

REPLICAT    ABENDED     REP1        00:00:00      00:20:15    

GGSCI (ahebsfin1) 3> start rep1

Sending START request to MANAGER ...

REPLICAT REP1 starting

GGSCI (ahebsfin1) 4> info all

Program     Status      Group       Lag at Chkpt  Time Since Chkpt

MANAGER     RUNNING                                           

REPLICAT    RUNNING     REP1        00:04:29      00:00:01    

Tray 

./ggsci: error while loading shared libraries: libclntshcore.so.12.1

PROBLEM:
---------------

While connecting goldengate console you may hit this error:


./ggsci: error while loading shared libraries: libclntshcore.so.12.1: cannot open shared object file: No such file or directory




SOLUTION:
-----------------

ln -s /u01/app/oracle/product/12.1.0.2/dbhome_1/lib/libclntshcore.so.12.1 libclntshcore.so.12.1





Note:Your ORACLE_HOME path may be diffrent.Attention please:)

ORA-15041: diskgroup "DG_NAME" space exhausted

In 11.1.0.7 database version;

When you want to add new datafile in ASM or resize the datafile which in ASM,you may hit this  error that ORA-15041: diskgroup "DG_NAME" space exhausted.

If ASM diskgroup can't rebalance this issue is normal.You can check disk free space with query below:

select group_number,free_mb from v$asm_disk order by 1;

An ideal disk group rebalance and each of the asm disks contains same free_mb.If it isn't same you hit a bug:)

Problem:
--------------

ORA-15041: diskgroup "DG_NAME" space exhausted

I hit an error diskgroup "DG_NAME" space exhausted.But I have free space on this diskgroup.

This is 11.1.0.7 bug

Solution(Workaround):
-----------------------------


alter system set "_asm_imbalance_tolerance"=0; 
alter diskgroup <DG_NAME> rebalance power 5; 



FREE_MB in Same Size Disks Differ Even After Successful Rebalance (Doc ID 813013.1)

ORA-00600: internal error code, arguments: [kccbcf_omf_reuse], [], [], [], []

PROBLEM:
__________

While creating controlfile in ASM,I hit this error below:

ERROR at line 1:
ORA-01503: CREATE CONTROLFILE failed
ORA-00600: internal error code, arguments: [kccbcf_omf_reuse], [], [], [], [],
[], [], []




SOLUTION:
__________

Control_files parameter set incorrectly in the spfile / pfile.

 *.control_files='+TEST/necodev/controlfile/current.ctl'

Bug 4309649: is fixed in 11g

Workaround:

 Use Diskgroup name 'only' for control_files init.ora parameter if create controlfile fails.
    Set the control_file parameter in the Auxiliary database spfile or pfile to 'only' the ASM Diskgroup name.

    Example:

     *.control_files='+TEST'


Best tray :))

ORA-12705: Cannot access NLS data files or invalid environment specified

PROBLEM:
----------------

While connecting sqlplus we got this error.

bash$>sqlplus A/A@NECODB


ERROR
-----------

ORA-12705: Cannot access NLS data files or invalid environment specified



SOLUTION:
----------------

You have to check ORACLE_HOME enviroment in your profile.

When we check the our .profile ,hit this line below:

export ORACLE_HOME=u101/app/oracle/product/10.2.0/db_1

This path is incorrect./ is missing

Must be /u101/app/oracle/product/10.2.0/db_1

After we changed the .profile.

Sqlplus connected database successfully .


Pray & Tray :)

CRS-1714:Unable to discover any voting files, retrying discovery

PROBLEM:
----------------

[cssd(26226)]CRS-1714:Unable to discover any voting files, retrying discovery in 15 seconds; Details at (:CSSNM00070:) in /u01/app/11.2.0/grid/log/ndx1/cssd/ocssd.log
2014-09-02 11:42:11.636:


SOLUTION:
----------------


****************************
Check the disks owner.                 *
Disks owner must be oracle          *
****************************

RMAN-20001: target database not found in recovery catalog

PROBLEM:
----------------

RMAN-06004: ORACLE error from recovery catalog database: RMAN-20001: target database not found in recovery catalog

SOLUTION:
-----------------

connect catalog database and register database to catalog

RMAN>register database


RMAN-20002: target database already registered in recovery catalog

ORA-12537:TNS:Connection Closed

PROBLEM:
----------------

ORA-12537:TNS:Connection Closed


SOLUTION:
---------------

Check the location which is oracle installed.

In my case,oracle software installed /oracle and /oracle %100.

So hit an "max number of process exceed" error in alertlog.

ORA-600 kdsgrp1

PROBLEM:
------------

Causes and Solutions for ora-600 [kdsgrp1] (Doc ID 1332252.1)


SOLUTION:
----------------


alter system flush buffer_cache;

Analyze table scott.dept validate structure cascade online;

ORA-01499: table/index cross reference failure

PROBLEM:
---------------

ORA-01499: table/index cross reference failure



SOLUTION:
-------------
Drop indexes

analyze table

create indexes

Asmlib ASM:/opt/oracle/extapi/64/asm/orcl/1/libasm.so op asm_open error Operation not permitted

PROBLEM:
----------------

asmlib ASM:/opt/oracle/extapi/64/asm/orcl/1/libasm.so op asm_open error Operation not permitted


SOLUTION:
--------------

To prevent ASMLIB from discovering the same disk more than once it needs to be forced to look only at /dev/dm*  and ignore all /dev/sd* devices.

1. either edit the file /etc/sysconfig/oracleasm-_dev_oracleasm    and change the lines:

ORACLEASM_SCANORDER=""
ORACLEASM_SCANEXCLUDE=""
to

ORACLEASM_SCANORDER="dm"
ORACLEASM_SCANEXCLUDE="sd"
or alternatively run the following command (as user root)

/usr/sbin/oracleasm configure -i -e -u user -g group -o "dm" -x "sd"

2. stop & restart ASMLIB as user root using:

/usr/sbin/oracleasm exit
/usr/sbin/oracleasm init
/usr/sbin/oracleasm scandisks

3. restart CRS or reboot node