Tuesday, July 22, 2025

Flashback data archive steps

 Objective: Ways to track DML changes in a table


Detailed objective:

We should be able to track the changes in the table at any point in time; say last 24hrs or last 12 hrs or between 2 different time frames



Options:

1. Trigger to fire on every insert or update or delete statements

2. Log miner, in case force logging is turned on.

3. DML tracking feature of oracle by default provides this info to some extent we can relate

4. Flashback data archive



Reference: https://blogs.oracle.com/connect/post/a-fresh-look-at-auditing-row-changes


Option 1:


Trigger [let us use most optimal setup - composite trigger]


Option 2:

Log Miner


Option 3:

DML tracking feature of oracle


select * from dba_Tab_modifications;


Option 4: Flashback data archive


Step 0)


Create the employees  table and load some data


CREATE TABLE EMPLOYEES (

    EMPLOYEE_ID    NUMBER(6),

    FIRST_NAME     VARCHAR2(20),

    LAST_NAME      VARCHAR2(25) NOT NULL,

    EMAIL          VARCHAR2(25) NOT NULL,

    PHONE_NUMBER   VARCHAR2(20),

    HIRE_DATE      DATE NOT NULL,

    JOB_ID         VARCHAR2(10) NOT NULL,

    SALARY         NUMBER(8,2),

    MANAGER_ID     NUMBER(6),

    DEPARTMENT_ID  NUMBER(4),

    VACATION_BALANCE NUMBER(6,2)

) tablespace tbspc_hr;



INSERT INTO EMPLOYEES (EMPLOYEE_ID, FIRST_NAME, LAST_NAME, EMAIL, PHONE_NUMBER, HIRE_DATE, JOB_ID, SALARY, MANAGER_ID, DEPARTMENT_ID, VACATION_BALANCE) VALUES (101, 'John', 'Doe', 'john.doe@example.com', '555-123-4567', TO_DATE('2020-01-15', 'YYYY-MM-DD'), 'IT_PROG', 60000.00, 100, 60, 20.50);

INSERT INTO EMPLOYEES (EMPLOYEE_ID, FIRST_NAME, LAST_NAME, EMAIL, PHONE_NUMBER, HIRE_DATE, JOB_ID, SALARY, MANAGER_ID, DEPARTMENT_ID, VACATION_BALANCE) VALUES (102, 'Jane', 'Smith', 'jane.smith@example.com', '555-987-6543', TO_DATE('2019-03-22', 'YYYY-MM-DD'), 'SA_REP', 85000.00, 101, 80, 25.00);

INSERT INTO EMPLOYEES (EMPLOYEE_ID, FIRST_NAME, LAST_NAME, EMAIL, PHONE_NUMBER, HIRE_DATE, JOB_ID, SALARY, MANAGER_ID, DEPARTMENT_ID, VACATION_BALANCE) VALUES (103, 'Peter', 'Jones', 'peter.jones@example.com', '555-111-2222', TO_DATE('2021-07-01', 'YYYY-MM-DD'), 'FI_ACCOUNT', 72000.00, 102, 90, 18.75);

INSERT INTO EMPLOYEES (EMPLOYEE_ID, FIRST_NAME, LAST_NAME, EMAIL, PHONE_NUMBER, HIRE_DATE, JOB_ID, SALARY, MANAGER_ID, DEPARTMENT_ID, VACATION_BALANCE) VALUES (104, 'Alice', 'Brown', 'alice.brown@example.com', '555-333-4444', TO_DATE('2018-11-10', 'YYYY-MM-DD'), 'AD_VP', 120000.00, NULL, 10, 30.00);

INSERT INTO EMPLOYEES (EMPLOYEE_ID, FIRST_NAME, LAST_NAME, EMAIL, PHONE_NUMBER, HIRE_DATE, JOB_ID, SALARY, MANAGER_ID, DEPARTMENT_ID, VACATION_BALANCE) VALUES (105, 'Robert', 'Davis', 'robert.davis@example.com', '555-555-6666', TO_DATE('2022-02-28', 'YYYY-MM-DD'), 'PU_CLERK', 35000.00, 103, 30, 15.00);

INSERT INTO EMPLOYEES (EMPLOYEE_ID, FIRST_NAME, LAST_NAME, EMAIL, PHONE_NUMBER, HIRE_DATE, JOB_ID, SALARY, MANAGER_ID, DEPARTMENT_ID, VACATION_BALANCE) VALUES (106, 'Maria', 'Garcia', 'maria.garcia@example.com', '555-777-8888', TO_DATE('2017-09-05', 'YYYY-MM-DD'), 'HR_REP', 50000.00, 104, 40, 22.00);

INSERT INTO EMPLOYEES (EMPLOYEE_ID, FIRST_NAME, LAST_NAME, EMAIL, PHONE_NUMBER, HIRE_DATE, JOB_ID, SALARY, MANAGER_ID, DEPARTMENT_ID, VACATION_BALANCE) VALUES (107, 'David', 'Miller', 'david.miller@example.com', '555-000-1111', TO_DATE('2023-04-18', 'YYYY-MM-DD'), 'MK_MAN', 95000.00, 101, 20, 28.00);

INSERT INTO EMPLOYEES (EMPLOYEE_ID, FIRST_NAME, LAST_NAME, EMAIL, PHONE_NUMBER, HIRE_DATE, JOB_ID, SALARY, MANAGER_ID, DEPARTMENT_ID, VACATION_BALANCE) VALUES (108, 'Sarah', 'Wilson', 'sarah.wilson@example.com', '555-222-3333', TO_DATE('2019-12-01', 'YYYY-MM-DD'), 'AC_MGR', 110000.00, 102, 100, 26.50);

INSERT INTO EMPLOYEES (EMPLOYEE_ID, FIRST_NAME, LAST_NAME, EMAIL, PHONE_NUMBER, HIRE_DATE, JOB_ID, SALARY, MANAGER_ID, DEPARTMENT_ID, VACATION_BALANCE) VALUES (109, 'Michael', 'Moore', 'michael.moore@example.com', '555-444-5555', TO_DATE('2020-06-20', 'YYYY-MM-DD'), 'SH_CLERK', 40000.00, 105, 50, 19.00);

INSERT INTO EMPLOYEES (EMPLOYEE_ID, FIRST_NAME, LAST_NAME, EMAIL, PHONE_NUMBER, HIRE_DATE, JOB_ID, SALARY, MANAGER_ID, DEPARTMENT_ID, VACATION_BALANCE) VALUES (110, 'Emily', 'Taylor', 'emily.taylor@example.com', '555-666-7777', TO_DATE('2021-01-01', 'YYYY-MM-DD'), 'PR_REP', 55000.00, 106, 70, 21.00);


commit;


Actual op:

SQL> create user hr identified by "hr";


User created.


SQL> grant create table, resource, create session, connect to hr;


Grant succeeded.


SQL> create tablespace tbspc_hr datafile size 10m autoextend on next 10m;


Tablespace created.


SQL> alter session set current_schema=HR;


Session altered.


SQL> alter user hr quota unlimited on tbspc_hr;


User altered.


SQL> CREATE TABLE EMPLOYEES (

    EMPLOYEE_ID    NUMBER(6),

    FIRST_NAME     VARCHAR2(20),

    LAST_NAME      VARCHAR2(25) NOT NULL,

    EMAIL          VARCHAR2(25) NOT NULL,

    PHONE_NUMBER   VARCHAR2(20),

    HIRE_DATE      DATE NOT NULL,

    JOB_ID         VARCHAR2(10) NOT NULL,

    SALARY         NUMBER(8,2),

    MANAGER_ID     NUMBER(6),

    DEPARTMENT_ID  NUMBER(4),

    VACATION_BALANCE NUMBER(6,2)

) tablespace tbspc_hr;  2    3    4    5    6    7    8    9   10   11   12   13


Table created.


SQL>


SQL> INSERT INTO EMPLOYEES (EMPLOYEE_ID, FIRST_NAME, LAST_NAME, EMAIL, PHONE_NUMBER, HIRE_DATE, JOB_ID, SALARY, MANAGER_ID, DEPARTMENT_ID, VACATION_BALANCE) VALUES (101, 'John', 'Doe', 'john.doe@example.com', '555-123-4567', TO_DATE('2020-01-15', 'YYYY-MM-DD'), 'IT_PROG', 60000.00, 100, 60, 20.50);


1 row created.


SQL> INSERT INTO EMPLOYEES (EMPLOYEE_ID, FIRST_NAME, LAST_NAME, EMAIL, PHONE_NUMBER, HIRE_DATE, JOB_ID, SALARY, MANAGER_ID, DEPARTMENT_ID, VACATION_BALANCE) VALUES (102, 'Jane', 'Smith', 'jane.smith@example.com', '555-987-6543', TO_DATE('2019-03-22', 'YYYY-MM-DD'), 'SA_REP', 85000.00, 101, 80, 25.00);


1 row created.


...


SQL> commit;


Commit complete.



Step 1) Create a tablespace to storage flashback data archive


create tablespace SPACE_FOR_ARCHIVE

datafile size 10M AUTOEXTEND ON NEXT 10M;


Output:

SQL> create tablespace SPACE_FOR_ARCHIVE

datafile size 10M AUTOEXTEND ON NEXT 10M;  2


Tablespace created.


SQL>


Step 2) Create a new structure to retain the information for a defined period (in our case 1 year)


CREATE FLASHBACK ARCHIVE longterm

TABLESPACE space_for_archive

RETENTION 1 YEAR;


Output:

SQL> CREATE FLASHBACK ARCHIVE longterm

TABLESPACE space_for_archive

RETENTION 1 YEAR;  2    3


Flashback archive created.


SQL> 


Step 3) Associate the needed table with the flashback archive.


ALTER TABLE EMPLOYEES FLASHBACK ARCHIVE LONGTERM;


Output:

SQL> ALTER TABLE EMPLOYEES FLASHBACK ARCHIVE LONGTERM;


Table altered.


SQL>


Step 4) Check all the tables in our current schema


select * from tab;


Step 5) Capture contextual info as well


exec dbms_flashback_archive.set_context_level('ALL');


Output:

SQL> exec dbms_flashback_archive.set_context_level('ALL');


PL/SQL procedure successfully completed.


SQL> 


Step 6) Check now the DML info on the table


select EMPLOYEE_ID, FIRST_NAME, VACATION_BALANCE,

       VERSIONS_STARTTIME TS,

       nvl(VERSIONS_OPERATION,'I') OP

--       dbms_flashback_archive.get_sys_context(versions_xid, 'USERENV','SESSION_USER') AS who,

--       dbms_flashback_archive.get_sys_context(versions_xid, 'USERENV','MODULE') AS program

from EMPLOYEES

versions between timestamp  timestamp '2025-07-22 01:03:00' and systimestamp

-- where EMPLOYEE_ID = 100

order by EMPLOYEE_ID, ts;


after initial loading:


SQL> select EMPLOYEE_ID, FIRST_NAME, VACATION_BALANCE,

       VERSIONS_STARTTIME TS,

       nvl(VERSIONS_OPERATION,'I') OP

--       dbms_flashback_archive.get_sys_context(versions_xid, 'USERENV','SESSION_USER') AS who,

--       dbms_flashback_archive.get_sys_context(versions_xid, 'USERENV','MODULE') AS program

from EMPLOYEES

versions between timestamp  timestamp '2025-07-22 01:03:00' and systimestamp

-- where EMPLOYEE_ID = 100

order by EMPLOYEE_ID, ts;  2    3    4    5    6    7    8    9


EMPLOYEE_ID FIRST_NAME           VACATION_BALANCE TS                                                                          O

----------- -------------------- ---------------- --------------------------------------------------------------------------- -

        101 Jack                             20.5 22-JUL-25 01.07.17 AM                                                       U

        101 John                             20.5                                                                             I

        102 Jane                               25                                                                             I

        103 Peter                           18.75                                                                             I

        104 Alice                              30                                                                             I

        105 Robert                             15                                                                             I

        106 Maria                              22                                                                             I

        107 David                              28                                                                             I

        108 Sarah                            26.5                                                                             I

        109 Michael                            19                                                                             I

        110 Emily                              21                                                                             I


11 rows selected.


SQL> 


upd:

update employees set FIRST_NAME='Jack' where EMPLOYEE_ID=101;


Output:

SQL> update EMPLOYEES set FIRST_NAME='Jack' where EMPLOYEE_ID=101;


1 row updated.


SQL> commit;


Commit complete.


SQL>


SQL> select EMPLOYEE_ID, FIRST_NAME, VACATION_BALANCE,

       VERSIONS_STARTTIME TS,

       nvl(VERSIONS_OPERATION,'I') OP

--       dbms_flashback_archive.get_sys_context(versions_xid, 'USERENV','SESSION_USER') AS who,

--       dbms_flashback_archive.get_sys_context(versions_xid, 'USERENV','MODULE') AS program

from EMPLOYEES

versions between timestamp  timestamp '2025-07-22 01:03:00' and systimestamp

-- where EMPLOYEE_ID = 100

order by EMPLOYEE_ID, ts;  2    3    4    5    6    7    8    9


EMPLOYEE_ID FIRST_NAME           VACATION_BALANCE TS                                                                          O

----------- -------------------- ---------------- --------------------------------------------------------------------------- -

        101 Jack                             20.5 22-JUL-25 01.07.17 AM                                                       U

        101 John                             20.5                                                                             I

        102 Jane                               25                                                                             I

        103 Peter                           18.75                                                                             I

        104 Alice                              30                                                                             I

        105 Robert                             15                                                                             I

        106 Maria                              22                                                                             I

        107 David                              28                                                                             I

        108 Sarah                            26.5                                                                             I

        109 Michael                            19                                                                             I

        110 Emily                              21                                                                             I


11 rows selected.


Now update 1 more record


select EMPLOYEE_ID, FIRST_NAME, VACATION_BALANCE,

       VERSIONS_STARTTIME TS,

       nvl(VERSIONS_OPERATION,'I') OP,

       dbms_flashback_archive.get_sys_context(versions_xid, 'USERENV','SESSION_USER') AS who,

       dbms_flashback_archive.get_sys_context(versions_xid, 'USERENV','MODULE') AS program

from EMPLOYEES

versions between timestamp  timestamp '2025-07-22 01:06:00' and systimestamp

-- where EMPLOYEE_ID = 100

where VERSIONS_STARTTIME  is not null

order by EMPLOYEE_ID, ts;


update employees set FIRST_NAME='Rose' where EMPLOYEE_ID=102;


Output:

01:10:13 SQL> update employees set FIRST_NAME='Rose' where EMPLOYEE_ID=102;


1 row updated.


Elapsed: 00:00:00.01

01:10:24 SQL> select EMPLOYEE_ID, FIRST_NAME, VACATION_BALANCE,

       VERSIONS_STARTTIME TS,

       nvl(VERSIONS_OPERATION,'I') OP

--       dbms_flashback_archive.get_sys_context(versions_xid, 'USERENV','SESSION_USER') AS who,

--       dbms_flashback_archive.get_sys_context(versions_xid, 'USERENV','MODULE') AS program

from EMPLOYEES

versions between timestamp  timestamp '2025-07-22 01:06:00' and systimestamp

-- where EMPLOYEE_ID = 100

where VERSIONS_STARTTIME  is not null

order by EMPLOYEE_ID, ts;01:10:29   2  01:10:29   3  01:10:29   4  01:10:29   5  01:10:29   6  01:10:29   7  01:10:29   8  01:10:29   9  01:10:29  10


EMPLOYEE_ID FIRST_NAME           VACATION_BALANCE TS                                                                          O

----------- -------------------- ---------------- --------------------------------------------------------------------------- -

        101 Jack                             20.5 22-JUL-25 01.07.17 AM                                                       U


Elapsed: 00:00:00.01

01:10:31 SQL> commit;


Commit complete.


Elapsed: 00:00:00.00

01:10:34 SQL>


01:10:36 SQL> select EMPLOYEE_ID, FIRST_NAME, VACATION_BALANCE,

       VERSIONS_STARTTIME TS,

       nvl(VERSIONS_OPERATION,'I') OP

--       dbms_flashback_archive.get_sys_context(versions_xid, 'USERENV','SESSION_USER') AS who,

01:10:41   2  01:10:41   3  01:10:41   4  01:10:41   5  --       dbms_flashback_archive.get_sys_context(versions_xid, 'USERENV','MODULE') AS program

from EMPLOYEES

versions between timestamp  timestamp '2025-07-22 01:06:00' and systimestamp

-- where EMPLOYEE_ID = 100

where VERSIONS_STARTTIME  is not null

order by EMPLOYEE_ID, ts;01:10:41   6  01:10:41   7  01:10:41   8  01:10:41   9  01:10:41  10


EMPLOYEE_ID FIRST_NAME           VACATION_BALANCE TS                                                                          O

----------- -------------------- ---------------- --------------------------------------------------------------------------- -

        101 Jack                             20.5 22-JUL-25 01.07.17 AM                                                       U

        102 Rose                               25 22-JUL-25 01.10.30 AM                                                       U


Elapsed: 00:00:00.01

01:10:42 SQL>


With context:


01:11:09 SQL> select EMPLOYEE_ID, FIRST_NAME, VACATION_BALANCE,

       VERSIONS_STARTTIME TS,

       nvl(VERSIONS_OPERATION,'I') OP,

       dbms_flashback_archive.get_sys_context(versions_xid, 'USERENV','SESSION_USER') AS who,

       dbms_flashback_archive.get_sys_context(versions_xid, 'USERENV','MODULE') AS program

from EMPLOYEES

versions between timestamp  timestamp '2025-07-22 01:06:00' and systimestamp

-- where EMPLOYEE_ID = 100

where VERSIONS_STARTTIME  is not null

order by EMPLOYEE_ID, ts;01:11:18   2  01:11:18   3  01:11:18   4  01:11:18   5  01:11:18   6  01:11:18   7  01:11:18   8  01:11:18   9  01:11:18  10


EMPLOYEE_ID FIRST_NAME           VACATION_BALANCE TS                                                                          O

----------- -------------------- ---------------- --------------------------------------------------------------------------- -

WHO

------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

PROGRAM

------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

        101 Jack                             20.5 22-JUL-25 01.07.17 AM                                                       U

SYS

sqlplus@vcentos79-oracle-ggtgt (TNS V1-V3)


        102 Rose                               25 22-JUL-25 01.10.30 AM                                                       U

SYS

sqlplus@vcentos79-oracle-ggtgt (TNS V1-V3)



Elapsed: 00:00:00.02

01:11:19 SQL>



>> All set


Objective: I have a table description available, but I need the table DDL for the same.

I can manually write it, ideally it will take 10 to 15mins. But using AI tools, we should be able to generate this no time.


Example:


Table1: EMPLOYEES

SQL> desc EMPLOYEES

Name                          Null?    Type

————————————————————————————— ———————— ————————————

EMPLOYEE_ID                            NUMBER(6)

FIRST_NAME                             VARCHAR2(20)

LAST_NAME                     NOT NULL VARCHAR2(25)

EMAIL                         NOT NULL VARCHAR2(25)

PHONE_NUMBER                           VARCHAR2(20)

HIRE_DATE                     NOT NULL DATE

JOB_ID                        NOT NULL VARCHAR2(10)

SALARY                                 NUMBER(8,2)

MANAGER_ID                             NUMBER(6)

DEPARTMENT_ID                          NUMBER(4)

VACATION_BALANCE                       NUMBER(6,2)


 Name                                      Null?    Type

 ----------------------------------------- -------- ----------------------------

 EMPLOYEE_ID                                        NUMBER(6)

 FIRST_NAME                                         VARCHAR2(20)

 LAST_NAME                                 NOT NULL VARCHAR2(25)

 EMAIL                                     NOT NULL VARCHAR2(25)

 PHONE_NUMBER                                       VARCHAR2(20)

 HIRE_DATE                                 NOT NULL DATE

 JOB_ID                                    NOT NULL VARCHAR2(10)

 SALARY                                             NUMBER(8,2)

 MANAGER_ID                                         NUMBER(6)

 DEPARTMENT_ID                                      NUMBER(4)

 VACATION_BALANCE                                   NUMBER(6,2)



DDL:

CREATE TABLE EMPLOYEES (

    EMPLOYEE_ID    NUMBER(6),

    FIRST_NAME     VARCHAR2(20),

    LAST_NAME      VARCHAR2(25) NOT NULL,

    EMAIL          VARCHAR2(25) NOT NULL,

    PHONE_NUMBER   VARCHAR2(20),

    HIRE_DATE      DATE NOT NULL,

    JOB_ID         VARCHAR2(10) NOT NULL,

    SALARY         NUMBER(8,2),

    MANAGER_ID     NUMBER(6),

    DEPARTMENT_ID  NUMBER(4),

    VACATION_BALANCE NUMBER(6,2)

);




Table2: AUDIT_EMPLOYEES

SQL> desc AUDIT_EMPLOYEES

Name                          Null?    Type

————————————————————————————— ———————— ————————————

AUD_WHO                                VARCHAR2(20)

AUD_WHEN                               DATE

AUD_OPERATION                          VARCHAR2(1)

AUD_MODULE                             VARCHAR2(30)

EMPLOYEE_ID                            NUMBER(6)

FIRST_NAME                             VARCHAR2(20)

LAST_NAME                     NOT NULL VARCHAR2(25)

EMAIL                         NOT NULL VARCHAR2(25)

PHONE_NUMBER                           VARCHAR2(20)

HIRE_DATE                     NOT NULL DATE

JOB_ID                        NOT NULL VARCHAR2(10)

SALARY                                 NUMBER(8,2)

MANAGER_ID                             NUMBER(6)

DEPARTMENT_ID                          NUMBER(4)

VACATION_BALANCE                       NUMBER(6,2)


 Name                                      Null?    Type

 ----------------------------------------- -------- ----------------------------

 AUD_WHO                                            VARCHAR2(20)

 AUD_WHEN                                           DATE

 AUD_OPERATION                                      VARCHAR2(1)

 AUD_MODULE                                         VARCHAR2(30)

 EMPLOYEE_ID                                        NUMBER(6)

 FIRST_NAME                                         VARCHAR2(20)

 LAST_NAME                                 NOT NULL VARCHAR2(25)

 EMAIL                                     NOT NULL VARCHAR2(25)

 PHONE_NUMBER                                       VARCHAR2(20)

 HIRE_DATE                                 NOT NULL DATE

 JOB_ID                                    NOT NULL VARCHAR2(10)

 SALARY                                             NUMBER(8,2)

 MANAGER_ID                                         NUMBER(6)

 DEPARTMENT_ID                                      NUMBER(4)

 VACATION_BALANCE                                   NUMBER(6,2)



Output:

CREATE TABLE AUDIT_EMPLOYEES (

    AUD_WHO          VARCHAR2(20),

    AUD_WHEN         DATE,

    AUD_OPERATION    VARCHAR2(1),

    AUD_MODULE       VARCHAR2(30),

    EMPLOYEE_ID      NUMBER(6),

    FIRST_NAME       VARCHAR2(20),

    LAST_NAME        VARCHAR2(25) NOT NULL,

    EMAIL            VARCHAR2(25) NOT NULL,

    PHONE_NUMBER     VARCHAR2(20),

    HIRE_DATE        DATE NOT NULL,

    JOB_ID           VARCHAR2(10) NOT NULL,

    SALARY           NUMBER(8,2),

    MANAGER_ID       NUMBER(6),

    DEPARTMENT_ID    NUMBER(4),

    VACATION_BALANCE NUMBER(6,2)

);




>> Result: Success. We generated the DDL from table description using Gemini.

Friday, July 18, 2025

Oracle Version 19.24 onwards: RMAN Progress Status Report

I was restoring a DB in  Oracle DB version 19.27, I noticed a new way oracle starting to dump the restore log. Below is an example content (sourced from internet):


This feature seems to be there from 19.24.
The log file gets updated normally in the begining like

Channel cx: restoring datafile 0004 to 'zzz' 
..

But after a while a message of the below starts printing


RMAN Progress Status Report at 20/05/2025 21:30:48
Progress for current step on channel: ch03 [INSTID = 1, SID = 5, SERIAL = 7654, Polling channel id= 150]
Type Total_Blocks Procd_Blocks Open_Time status setcount File_Name
IN 0 734862848 20/05/2025 20:02:59 IN PROGRESS 43126 76888_TESTDB_3xxxx_121_1_1
OUT 3932160 1679871 20/05/2025 21:35:58 IN PROGRESS 43126 ..
OUT 3932160 1679871 20/05/2025 21:35:58 IN PROGRESS 43126 ..
OUT 3932160 1679871 20/05/2025 21:35:58 IN PROGRESS 43126 ..


A little nice feature.

Thanks


..

Commvault as backup application - HP StoreOnce as Media : OSCLT_ERR_NO_DEVICES_DISCOVERED

 Observation: Once I was conducting a full db restore, 1/3 of my db files were restored then all my 6 streams failed due to below error


The error below is sourced from internet to show you the example error, all the error codes are same just the media name will be different

channel ch1: reading from backup piece 61xxxxxxxxxxx
ORA-19870: error reading backup piece 61xxxxxxxxxxx
ORA-19507: failed to retrieve sequential file, handle="61xxxxxxxxxxx", parms=""
ORA-27029: skgfrtrv: sbtrestore returned error
ORA-19511: Error received from media manager layer, error text:
sbtrestore: Job[44240] thread[3696]: CVArchive->open() failed.


What should you do in such error?

Have a call with your backup team, have them display the commvault restore job 44240 log.

If you notice the job failed due to an error in the subject of the blog, then you likely to face a communication error between commvault and storeonce device.

Mostly due to insufficient resources.

In such cases, please restart your restore after informing your backup team to keep an eye on the restore and resource usage in both StoreOnce and Commvault.


Remember StoreOnce has a limit to the number of active streams it can handle, I believe it will be 200 or some number. When that is exceeded we can face communication issue like above.


Thanks

Tuesday, July 8, 2025

Fix for ORA-01031 TTS EXPDP with database vault

Grant the below permission to system user 

1. EXP_FULL_DATABASE

2. IMP_FULL_DATABASE

3. exec dvsys.dbms_macadm.authorize_datapump_user('SYSTEM');

4. grant dv_owner to system.


This should handle the full=y and transportable=always scenraio in expdp just fine.

It is tested in 19.27.

Saturday, June 21, 2025

ORA-28376: cannot find PKCS11 library

 Verify permission for 


cd /opt

find oracle -exec ls -ld {} \;

See if all the directories are owned by root and it takes read to world

If so fix it; by granting 0755 to all the sub directories from /opt/oracle/....

proper permission:


[oracle@vcentos79-oracle-ggtgt ~]$ find /opt/oracle -exec ls -ld {} \;

drwxr-xr-x. 3 root root 20 Sep  6  2024 /opt/oracle

drwxr-xr-x. 3 root root 16 Sep  6  2024 /opt/oracle/extapi

drwxr-xr-x. 3 root root 17 Sep  6  2024 /opt/oracle/extapi/64

drwxr-xr-x. 3 root root 20 Sep  6  2024 /opt/oracle/extapi/64/hsm

drwxr-xr-x. 3 root root 19 Sep  6  2024 /opt/oracle/extapi/64/hsm/oracle

drwxr-xr-x. 2 root root 27 Sep  6  2024 /opt/oracle/extapi/64/hsm/oracle/1.0.0

-rwxr-xr-x. 1 root root 8924633 Sep  6  2024 /opt/oracle/extapi/64/hsm/oracle/1.0.0/liborapkcs.so

[oracle@vcentos79-oracle-ggtgt ~]$


Friday, June 13, 2025

How to draw interactive diagram using draw.io

How to draw an interactive diagram in draw.io website

Objective: Use draw.io web tool to create an interactive diagram to demonstrate the technical solution to the teams

url: https://app.diagrams.net/ [aka draw.io]
custom code: https://jgraph.github.io/drawio-tools/tools/link.html

Our goal: Show data migration options for oracle db

About draw.io:
security first tool.
Always store your document or presentation or diagram on your local machine or your personal drive.
Downloadable desktop app is available on their website. So you can work offline.

The content is stored on draw.io server only when you convert the files to image, pdf etc...
But they will be deleted is what they claim as soon as the processing is done.

I still use it without any identity information documented in draw.io.

So steps for interactive diagram:

step1: hit to the url
step2: create your diagram
step3: make it interactive by creation actions using custom links (ctrl+k & json code)
step4: Publish the file for presentation by going to files menu

Enjoy an interactive diagram you were looking for and be smart.

Thanks

Sunday, April 6, 2025

Setup AI agent browser-use using web-ui from scratch

 

Setup AI agent browser-use using  web-ui from scratch


Notice here, I dont want to don anything on my local laptop, since this will mess up my local permenant python setup. So I use a VM.


references:

https://docs.astral.sh/uv/

https://github.com/browser-use/browser-use?tab=readme-ov-file

https://docs.browser-use.com/quickstart


YouTube:


                        

1. setup chrome as your browser


2. install uv as package manager [a package and project manager written in rust]


https://docs.astral.sh/uv/getting-started/installation/



Examine the script:


powershell -c "irm https://astral.sh/uv/install.ps1 | more"


select windows install option:


powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"


> copy/paste the codes above


Output:


Install the latest PowerShell for new features and improvements! https://aka.ms/PSWindows                                                                                                                                                                                                                                                                                                                                                                                                 PS C:\Users\vagrant> powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"                                                                                                                                      Downloading uv 0.6.12 (x86_64-pc-windows-msvc)                                                                                                                                                                                               Installing to C:\Users\vagrant\.local\bin

  uv.exe

  uvx.exe

everything's installed!


To add C:\Users\vagrant\.local\bin to your PATH, either restart your shell or run:


    set Path=C:\Users\vagrant\.local\bin;%Path%   (cmd)

    $env:Path = "C:\Users\vagrant\.local\bin;$env:Path"   (powershell)

PS C:\Users\vagrant>




3. Install git


https://git-scm.com/downloads/win


4. Download web-ui repo and setup the browser-use repo on local laptop

cmd

cd c:\user\vagrant

mkdir web-ui-agent

cd web-ui-agent

dir

git clone https://github.com/browser-use/web-ui.git

cd web-ui



output:

C:\Users\vagrant>cd web-ui-agent


C:\Users\vagrant\web-ui-agent>dir

 Volume in drive C has no label.

 Volume Serial Number is 9499-6C73


 Directory of C:\Users\vagrant\web-ui-agent


04/05/2025  12:45 AM    <DIR>          .

04/05/2025  12:49 AM    <DIR>          ..

               0 File(s)              0 bytes

               2 Dir(s)  103,294,328,832 bytes free


C:\Users\vagrant\web-ui-agent>git clone https://github.com/browser-use/web-ui.git

Cloning into 'web-ui'...

remote: Enumerating objects: 1653, done.

remote: Counting objects: 100% (685/685), done.

remote: Compressing objects: 100% (82/82), done.

remote: Total 1653 (delta 627), reused 603 (delta 603), pack-reused 968 (from 2)

Receiving objects: 100% (1653/1653), 984.22 KiB | 5.89 MiB/s, done.

Resolving deltas: 100% (1028/1028), done.


C:\Users\vagrant\web-ui-agent>dir

 Volume in drive C has no label.

 Volume Serial Number is 9499-6C73


 Directory of C:\Users\vagrant\web-ui-agent


04/05/2025  01:12 AM    <DIR>          .

04/05/2025  12:49 AM    <DIR>          ..

04/05/2025  01:12 AM    <DIR>          web-ui

               0 File(s)              0 bytes

               3 Dir(s)  103,291,965,440 bytes free


C:\Users\vagrant\web-ui-agent>cd web-ui


C:\Users\vagrant\web-ui-agent\web-ui>dir

 Volume in drive C has no label.

 Volume Serial Number is 9499-6C73


 Directory of C:\Users\vagrant\web-ui-agent\web-ui


04/05/2025  01:12 AM    <DIR>          .

04/05/2025  01:12 AM    <DIR>          ..

04/05/2025  01:12 AM                 9 .dockerignore

04/05/2025  01:12 AM             1,286 .env.example

04/05/2025  01:12 AM             3,603 .gitignore

04/05/2025  01:12 AM    <DIR>          .vscode

04/05/2025  01:12 AM    <DIR>          assets

04/05/2025  01:12 AM             2,426 docker-compose.yml

04/05/2025  01:12 AM             2,084 Dockerfile

04/05/2025  01:12 AM             2,112 Dockerfile.arm64

04/05/2025  01:12 AM               160 entrypoint.sh

04/05/2025  01:12 AM             1,094 LICENSE

04/05/2025  01:12 AM             9,506 README.md

04/05/2025  01:12 AM               156 requirements.txt

04/05/2025  01:12 AM             1,052 SECURITY.md

04/05/2025  01:12 AM    <DIR>          src

04/05/2025  01:12 AM             3,051 supervisord.conf

04/05/2025  01:12 AM    <DIR>          tests

04/05/2025  01:12 AM            48,145 webui.py

              13 File(s)         74,684 bytes

               6 Dir(s)  103,291,965,440 bytes free


C:\Users\vagrant\web-ui-agent\web-ui>




5. Download web-ui repo and setup the browser-use repo on local laptop



uv python list

uv python install 

uv python list


Output:


c:\Users\vagrant\web-ui-agent\web-ui>uv python list

cpython-3.14.0a6+freethreaded-windows-x86_64-none    <download available>

cpython-3.14.0a6-windows-x86_64-none                 <download available>

cpython-3.13.2+freethreaded-windows-x86_64-none      <download available>

cpython-3.13.2-windows-x86_64-none                   <download available>

cpython-3.12.9-windows-x86_64-none                   <download available>

cpython-3.11.11-windows-x86_64-none                  <download available>

cpython-3.10.16-windows-x86_64-none                  <download available>

cpython-3.9.21-windows-x86_64-none                   <download available>

cpython-3.8.20-windows-x86_64-none                   <download available>

cpython-3.7.9-windows-x86_64-none                    <download available>


C:\Users\vagrant\web-ui-agent\web-ui>uv python install

Installed Python 3.13.2 in 5.21s

 + cpython-3.13.2-windows-x86_64-none


C:\Users\vagrant\web-ui-agent\web-ui>uv python list

cpython-3.14.0a6+freethreaded-windows-x86_64-none    <download available>

cpython-3.14.0a6-windows-x86_64-none                 <download available>

cpython-3.13.2+freethreaded-windows-x86_64-none      <download available>

cpython-3.13.2-windows-x86_64-none                   C:\Users\vagrant\AppData\Roaming\uv\python\cpython-3.13.2-windows-x86_64-none\python.exe

cpython-3.12.9-windows-x86_64-none                   <download available>

cpython-3.11.11-windows-x86_64-none                  <download available>

cpython-3.10.16-windows-x86_64-none                  <download available>

cpython-3.9.21-windows-x86_64-none                   <download available>

cpython-3.8.20-windows-x86_64-none                   <download available>

cpython-3.7.9-windows-x86_64-none                    <download available>


C:\Users\vagrant\web-ui-agent\web-ui>




6. Install Python, activate python virtual environment & install all dependent packages using uv


uv venv --python 3.13.2

.venv\Scripts\activate

uv pip list

uv pip install -r requirements.txt

uv pip list


output:

(web-ui) C:\Users\vagrant\web-ui-agent\web-ui>uv pip install -r requirements.txt

Resolved 122 packages in 4.80s

      Built html2text==2024.2.26

Prepared 22 packages in 10.84s

Installed 122 packages in 4.17s

 + aiofiles==23.2.1

 + annotated-types==0.7.0

 + anthropic==0.49.0

 + anyio==4.9.0

 + audioop-lts==0.2.1

 + babel==2.17.0

 + backoff==2.2.1

 + beautifulsoup4==4.13.3

 + browser-use==0.1.40

 + cachetools==5.5.2

 + certifi==2025.1.31

 + charset-normalizer==3.4.1

 + click==8.1.8

 + colorama==0.4.6

 + courlan==1.3.2

 + dateparser==1.2.1

 + defusedxml==0.7.1

 + distro==1.9.0

 + fastapi==0.115.12

 + ffmpy==0.5.0

 + filelock==3.18.0

 + filetype==1.2.0

 + fsspec==2025.3.2

 + google-ai-generativelanguage==0.6.15

 + google-api-core==2.24.2

 + google-api-python-client==2.166.0

 + google-auth==2.38.0

 + google-auth-httplib2==0.2.0

 + google-generativeai==0.8.4

 + googleapis-common-protos==1.69.2

 + gradio==5.23.1

 + gradio-client==1.8.0

 + greenlet==3.1.1

 + groovy==0.1.2

 + grpcio==1.71.0

 + grpcio-status==1.71.0

 + h11==0.14.0

 + html2text==2024.2.26

 + htmldate==1.9.3

 + httpcore==1.0.7

 + httplib2==0.22.0

 + httpx==0.28.1

 + httpx-sse==0.4.0

 + huggingface-hub==0.30.1

 + idna==3.10

 + jinja2==3.1.6

 + jiter==0.9.0

 + json-repair==0.40.0

 + jsonpatch==1.33

 + jsonpointer==3.0.0

 + justext==3.0.2

 + langchain-anthropic==0.3.3

 + langchain-core==0.3.51

 + langchain-google-genai==2.0.8

 + langchain-mistralai==0.2.4

 + langchain-ollama==0.2.2

 + langchain-openai==0.3.1

 + langsmith==0.3.24

 + lxml==5.3.1

 + lxml-html-clean==0.4.1

 + maincontentextractor==0.0.4

 + markdown-it-py==3.0.0

 + markdownify==0.14.1

 + markupsafe==3.0.2

 + mdurl==0.1.2

 + monotonic==1.6

 + numpy==2.2.4

 + ollama==0.4.7

 + openai==1.70.0

 + orjson==3.10.16

 + packaging==24.2

 + pandas==2.2.3

 + pillow==11.1.0

 + playwright==1.51.0

 + posthog==3.23.0

 + proto-plus==1.26.1

 + protobuf==5.29.4

 + pyasn1==0.6.1

 + pyasn1-modules==0.4.2

 + pydantic==2.11.2

 + pydantic-core==2.33.1

 + pydub==0.25.1

 + pyee==12.1.1

 + pygments==2.19.1

 + pyparsing==3.2.3

 + pyperclip==1.9.0

 + python-dateutil==2.9.0.post0

 + python-dotenv==1.1.0

 + python-multipart==0.0.20

 + pytz==2025.2

 + pyyaml==6.0.2

 + regex==2024.11.6

 + requests==2.32.3

 + requests-toolbelt==1.0.0

 + rich==14.0.0

 + rsa==4.9

 + ruff==0.11.4

 + safehttpx==0.1.6

 + semantic-version==2.10.0

 + setuptools==78.1.0

 + shellingham==1.5.4

 + six==1.17.0

 + sniffio==1.3.1

 + soupsieve==2.6

 + starlette==0.46.1

 + tenacity==9.1.2

 + tiktoken==0.9.0

 + tld==0.13

 + tokenizers==0.21.1

 + tomlkit==0.13.2

 + tqdm==4.67.1

 + trafilatura==2.0.0

 + typer==0.15.2

 + typing-extensions==4.13.1

 + typing-inspection==0.4.0

 + tzdata==2025.2

 + tzlocal==5.3.1

 + uritemplate==4.1.1

 + urllib3==2.3.0

 + uvicorn==0.34.0

 + websockets==15.0.1

 + zstandard==0.23.0


(web-ui) C:\Users\vagrant\web-ui-agent\web-ui>


(web-ui) C:\Users\vagrant\web-ui-agent\web-ui>uv pip list

Package                      Version

---------------------------- -----------

aiofiles                     23.2.1

annotated-types              0.7.0

anthropic                    0.49.0

anyio                        4.9.0

audioop-lts                  0.2.1

babel                        2.17.0

backoff                      2.2.1

beautifulsoup4               4.13.3

browser-use                  0.1.40

cachetools                   5.5.2

certifi                      2025.1.31

charset-normalizer           3.4.1

click                        8.1.8

colorama                     0.4.6

courlan                      1.3.2

dateparser                   1.2.1

defusedxml                   0.7.1

distro                       1.9.0

fastapi                      0.115.12

ffmpy                        0.5.0

filelock                     3.18.0

filetype                     1.2.0

fsspec                       2025.3.2

google-ai-generativelanguage 0.6.15

google-api-core              2.24.2

google-api-python-client     2.166.0

google-auth                  2.38.0

google-auth-httplib2         0.2.0

google-generativeai          0.8.4

googleapis-common-protos     1.69.2

gradio                       5.23.1

gradio-client                1.8.0

greenlet                     3.1.1

groovy                       0.1.2

grpcio                       1.71.0

grpcio-status                1.71.0

h11                          0.14.0

html2text                    2024.2.26

htmldate                     1.9.3

httpcore                     1.0.7

httplib2                     0.22.0

httpx                        0.28.1

httpx-sse                    0.4.0

huggingface-hub              0.30.1

idna                         3.10

jinja2                       3.1.6

jiter                        0.9.0

json-repair                  0.40.0

jsonpatch                    1.33

jsonpointer                  3.0.0

justext                      3.0.2

langchain-anthropic          0.3.3

langchain-core               0.3.51

langchain-google-genai       2.0.8

langchain-mistralai          0.2.4

langchain-ollama             0.2.2

langchain-openai             0.3.1

langsmith                    0.3.24

lxml                         5.3.1

lxml-html-clean              0.4.1

maincontentextractor         0.0.4

markdown-it-py               3.0.0

markdownify                  0.14.1

markupsafe                   3.0.2

mdurl                        0.1.2

monotonic                    1.6

numpy                        2.2.4

ollama                       0.4.7

openai                       1.70.0

orjson                       3.10.16

packaging                    24.2

pandas                       2.2.3

pillow                       11.1.0

playwright                   1.51.0

posthog                      3.23.0

proto-plus                   1.26.1

protobuf                     5.29.4

pyasn1                       0.6.1

pyasn1-modules               0.4.2

pydantic                     2.11.2

pydantic-core                2.33.1

pydub                        0.25.1

pyee                         12.1.1

pygments                     2.19.1

pyparsing                    3.2.3

pyperclip                    1.9.0

python-dateutil              2.9.0.post0

python-dotenv                1.1.0

python-multipart             0.0.20

pytz                         2025.2

pyyaml                       6.0.2

regex                        2024.11.6

requests                     2.32.3

requests-toolbelt            1.0.0

rich                         14.0.0

rsa                          4.9

ruff                         0.11.4

safehttpx                    0.1.6

semantic-version             2.10.0

setuptools                   78.1.0

shellingham                  1.5.4

six                          1.17.0

sniffio                      1.3.1

soupsieve                    2.6

starlette                    0.46.1

tenacity                     9.1.2

tiktoken                     0.9.0

tld                          0.13

tokenizers                   0.21.1

tomlkit                      0.13.2

tqdm                         4.67.1

trafilatura                  2.0.0

typer                        0.15.2

typing-extensions            4.13.1

typing-inspection            0.4.0

tzdata                       2025.2

tzlocal                      5.3.1

uritemplate                  4.1.1

urllib3                      2.3.0

uvicorn                      0.34.0

websockets                   15.0.1

zstandard                    0.23.0


(web-ui) C:\Users\vagrant\web-ui-agent\web-ui>



7. Install headless browser chromium:

playwright install --with-deps chromium


Output:

(web-ui) C:\Users\vagrant\web-ui-agent\web-ui>playwright install --with-deps chromium


Success Restart Needed Exit Code      Feature Result

------- -------------- ---------      --------------

True    No             Success        {Media Foundation}



8. Copy environment file containing google’s gemini flash llm api key:

copy <>\.env .env



Output:


(web-ui) C:\Users\vagrant\web-ui-agent\web-ui>


copy <>\.env .env


>> done


Directory of C:\Users\vagrant\web-ui-agent\web-ui


04/05/2025  01:24 AM    <DIR>          .

04/05/2025  01:12 AM    <DIR>          ..

04/05/2025  01:12 AM                 9 .dockerignore

04/04/2025  01:36 AM               153 .env << file exists

04/05/2025  01:12 AM             1,286 .env.example

04/05/2025  01:12 AM             3,603 .gitignore

04/05/2025  01:16 AM    <DIR>          .venv

04/05/2025  01:12 AM    <DIR>          .vscode

04/05/2025  01:12 AM    <DIR>          assets

04/05/2025  01:12 AM             2,426 docker-compose.yml

04/05/2025  01:12 AM             2,084 Dockerfile

04/05/2025  01:12 AM             2,112 Dockerfile.arm64

04/05/2025  01:12 AM               160 entrypoint.sh

04/05/2025  01:12 AM             1,094 LICENSE

04/05/2025  01:12 AM             9,506 README.md

04/05/2025  01:12 AM               156 requirements.txt

04/05/2025  01:12 AM             1,052 SECURITY.md

04/05/2025  01:12 AM    <DIR>          src

04/05/2025  01:12 AM             3,051 supervisord.conf

04/05/2025  01:12 AM    <DIR>          tests

04/05/2025  01:12 AM            48,145 webui.py

              14 File(s)         74,837 bytes

               7 Dir(s)  102,828,404,736 bytes free


(web-ui) C:\Users\vagrant\web-ui-agent\web-ui>



9. Launch the webui program:


python webui.py --ip 127.0.0.1 --port 7788


Launch webui: http://127.0.0.1:7788






Flashback data archive steps

 Objective: Ways to track DML changes in a table Detailed objective: We should be able to track the changes in the table at any point in tim...