Showing posts with label postgresql 15. Show all posts
Showing posts with label postgresql 15. Show all posts

Tuesday, October 25, 2022

postgresql physical backup - BARMAN

Objective: In this blog, we will examine the steps involved in barman backup configuration.



BARMAN has to be installed in a seperate server than the postgresql server.
Since my machine is very small in size, I installed barman on the same machine as postgresql server.
As per instruction from https://www.digitalocean.com/community/tutorials/how-to-back-up-restore-and-migrate-postgresql-databases-with-barman-on-centos-7
We already have postgresql server installed (version 15). Also we have epel and postgresql yum repos needed for this install step (see the packages downloaded were from either base, epel or pgdg-common).
We just need to install barman using yum install...

Actual output:

[root@10 ~]# yum install barman
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
epel/x86_64/metalink                                                                               | 6.1 kB  00:00:00
 * base: centos.excellmedia.net
 * epel: mirrors.bestthaihost.com
 * extras: centos.excellmedia.net
 * updates: centos.excellmedia.net
base                                                                                               | 3.6 kB  00:00:00
epel                                                                                               | 4.7 kB  00:00:00
extras                                                                                             | 2.9 kB  00:00:00
pgdg-common/7/x86_64/signature                                                                     |  198 B  00:00:00
pgdg-common/7/x86_64/signature                                                                     | 2.9 kB  00:00:00 !!!
pgdg10/7/x86_64/signature                                                                          |  198 B  00:00:00
pgdg10/7/x86_64/signature                                                                          | 3.6 kB  00:00:00 !!!
pgdg11/7/x86_64/signature                                                                          |  198 B  00:00:00
pgdg11/7/x86_64/signature                                                                          | 3.6 kB  00:00:00 !!!
pgdg12/7/x86_64/signature                                                                          |  198 B  00:00:00
pgdg12/7/x86_64/signature                                                                          | 3.6 kB  00:00:00 !!!
pgdg13/7/x86_64/signature                                                                          |  198 B  00:00:00
pgdg13/7/x86_64/signature                                                                          | 3.6 kB  00:00:00 !!!
pgdg14/7/x86_64/signature                                                                          |  198 B  00:00:00
pgdg14/7/x86_64/signature                                                                          | 3.6 kB  00:00:00 !!!
pgdg15/7/x86_64/signature                                                                          |  198 B  00:00:00
pgdg15/7/x86_64/signature                                                                          | 3.6 kB  00:00:00 !!!
updates                                                                                            | 2.9 kB  00:00:00
(1/8): pgdg10/7/x86_64/primary_db                                                                  | 396 kB  00:00:12
(2/8): pgdg11/7/x86_64/primary_db                                                                  | 440 kB  00:00:13
(3/8): pgdg13/7/x86_64/primary_db                                                                  | 224 kB  00:00:02
(4/8): pgdg14/7/x86_64/primary_db                                                                  | 146 kB  00:00:01
(5/8): pgdg15/7/x86_64/primary_db                                                                  |  56 kB  00:00:00
(6/8): pgdg12/7/x86_64/primary_db                                                                  | 326 kB  00:00:17
(7/8): epel/x86_64/primary_db                                                                      | 7.0 MB  00:00:29
(8/8): epel/x86_64/updateinfo                                                                      | 1.0 MB  00:04:03
Resolving Dependencies
--> Running transaction check
---> Package barman.noarch 0:3.2.0-1.rhel7 will be installed
--> Processing Dependency: python-barman = 3.2.0 for package: barman-3.2.0-1.rhel7.noarch
--> Processing Dependency: rsync >= 3.0.4 for package: barman-3.2.0-1.rhel7.noarch
--> Running transaction check
---> Package python-barman.noarch 0:3.2.0-1.rhel7 will be installed
--> Processing Dependency: python-psycopg2 >= 2.4.2 for package: python-barman-3.2.0-1.rhel7.noarch
--> Processing Dependency: python-setuptools for package: python-barman-3.2.0-1.rhel7.noarch
--> Processing Dependency: python-dateutil for package: python-barman-3.2.0-1.rhel7.noarch
--> Processing Dependency: python-argcomplete for package: python-barman-3.2.0-1.rhel7.noarch
---> Package rsync.x86_64 0:3.1.2-11.el7_9 will be installed
--> Running transaction check
---> Package python-dateutil.noarch 0:1.5-7.el7 will be installed
---> Package python-setuptools.noarch 0:0.9.8-7.el7 will be installed
--> Processing Dependency: python-backports-ssl_match_hostname for package: python-setuptools-0.9.8-7.el7.noarch
---> Package python2-argcomplete.noarch 0:1.7.0-4.el7 will be installed
---> Package python2-psycopg2.x86_64 0:2.8.6-1.rhel7 will be installed
--> Running transaction check
---> Package python-backports-ssl_match_hostname.noarch 0:3.5.0.1-1.el7 will be installed
--> Processing Dependency: python-ipaddress for package: python-backports-ssl_match_hostname-3.5.0.1-1.el7.noarch
--> Processing Dependency: python-backports for package: python-backports-ssl_match_hostname-3.5.0.1-1.el7.noarch
--> Running transaction check
---> Package python-backports.x86_64 0:1.0-8.el7 will be installed
---> Package python-ipaddress.noarch 0:1.0.16-2.el7 will be installed
--> Finished Dependency Resolution
Dependencies Resolved
==========================================================================================================================
 Package                                        Arch              Version                    Repository              Size
==========================================================================================================================
Installing:
 barman                                         noarch            3.2.0-1.rhel7              pgdg-common             44 k
Installing for dependencies:
 python-backports                               x86_64            1.0-8.el7                  base                   5.8 k
 python-backports-ssl_match_hostname            noarch            3.5.0.1-1.el7              base                    13 k
 python-barman                                  noarch            3.2.0-1.rhel7              pgdg-common            451 k
 python-dateutil                                noarch            1.5-7.el7                  base                    85 k
 python-ipaddress                               noarch            1.0.16-2.el7               base                    34 k
 python-setuptools                              noarch            0.9.8-7.el7                base                   397 k
 python2-argcomplete                            noarch            1.7.0-4.el7                epel                    49 k
 python2-psycopg2                               x86_64            2.8.6-1.rhel7              pgdg-common            171 k
 rsync                                          x86_64            3.1.2-11.el7_9             updates                408 k
Transaction Summary
==========================================================================================================================
Install  1 Package (+9 Dependent packages)
Total download size: 1.6 M
Installed size: 6.5 M
Is this ok [y/d/N]: y
Downloading packages:
(1/10): python-backports-1.0-8.el7.x86_64.rpm                                                      | 5.8 kB  00:00:06
(2/10): python-backports-ssl_match_hostname-3.5.0.1-1.el7.noarch.rpm                               |  13 kB  00:00:06
(3/10): python-dateutil-1.5-7.el7.noarch.rpm                                                       |  85 kB  00:00:07
(4/10): barman-3.2.0-1.rhel7.noarch.rpm                                                            |  44 kB  00:00:08
(5/10): python2-psycopg2-2.8.6-1.rhel7.x86_64.rpm                                                  | 171 kB  00:00:04
(6/10): python-ipaddress-1.0.16-2.el7.noarch.rpm                                                   |  34 kB  00:00:06
(7/10): python-setuptools-0.9.8-7.el7.noarch.rpm                                                   | 397 kB  00:00:08
(8/10): python-barman-3.2.0-1.rhel7.noarch.rpm                                                     | 451 kB  00:00:16
(9/10): python2-argcomplete-1.7.0-4.el7.noarch.rpm                                                 |  49 kB  00:00:08
(10/10): rsync-3.1.2-11.el7_9.x86_64.rpm                                                           | 408 kB  00:00:06
--------------------------------------------------------------------------------------------------------------------------
Total                                                                                      84 kB/s | 1.6 MB  00:00:19
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Installing : python2-psycopg2-2.8.6-1.rhel7.x86_64                                                                 1/10
  Installing : python-ipaddress-1.0.16-2.el7.noarch                                                                  2/10
  Installing : python-backports-1.0-8.el7.x86_64                                                                     3/10
  Installing : python-backports-ssl_match_hostname-3.5.0.1-1.el7.noarch                                              4/10
  Installing : python-setuptools-0.9.8-7.el7.noarch                                                                  5/10
  Installing : python2-argcomplete-1.7.0-4.el7.noarch                                                                6/10
  Installing : python-dateutil-1.5-7.el7.noarch                                                                      7/10
  Installing : python-barman-3.2.0-1.rhel7.noarch                                                                    8/10
  Installing : rsync-3.1.2-11.el7_9.x86_64                                                                           9/10
  Installing : barman-3.2.0-1.rhel7.noarch                                                                          10/10
  Verifying  : rsync-3.1.2-11.el7_9.x86_64                                                                           1/10
  Verifying  : python-backports-ssl_match_hostname-3.5.0.1-1.el7.noarch                                              2/10
  Verifying  : python-dateutil-1.5-7.el7.noarch                                                                      3/10
  Verifying  : python2-argcomplete-1.7.0-4.el7.noarch                                                                4/10
  Verifying  : barman-3.2.0-1.rhel7.noarch                                                                           5/10
  Verifying  : python-backports-1.0-8.el7.x86_64                                                                     6/10
  Verifying  : python-ipaddress-1.0.16-2.el7.noarch                                                                  7/10
  Verifying  : python2-psycopg2-2.8.6-1.rhel7.x86_64                                                                 8/10
  Verifying  : python-barman-3.2.0-1.rhel7.noarch                                                                    9/10
  Verifying  : python-setuptools-0.9.8-7.el7.noarch                                                                 10/10
Installed:
  barman.noarch 0:3.2.0-1.rhel7
Dependency Installed:
  python-backports.x86_64 0:1.0-8.el7                python-backports-ssl_match_hostname.noarch 0:3.5.0.1-1.el7
  python-barman.noarch 0:3.2.0-1.rhel7               python-dateutil.noarch 0:1.5-7.el7
  python-ipaddress.noarch 0:1.0.16-2.el7             python-setuptools.noarch 0:0.9.8-7.el7
  python2-argcomplete.noarch 0:1.7.0-4.el7           python2-psycopg2.x86_64 0:2.8.6-1.rhel7
  rsync.x86_64 0:3.1.2-11.el7_9
Complete!

[root@10 ~]# id barman
uid=998(barman) gid=996(barman) groups=996(barman)
[root@10 ~]#

We alrady have the pg_hba.conf with the below entry, this means that connections coming from 127.0.0.1/32 with all users by default will be allowed without password prompt.

# IPv4 local connections:
host    all             all             127.0.0.1/32            trust

Let us setup passwordless ssh setup between barman and postgres user...
The passwordless was setup using the below command

ssh-keygen -t rsa

The command has to be run as both postgres and barman user. The .pub key of both the users needs to be put into authorized_keys file of the .ssh directories of the corresponding user.

Still I faced issues with postgres user (postgres user's password was always prompted), mistake I made was I copied authorized_keys file from barman, it messed up my selinux context for ever... I try remove and readd of the authorized_keys file in the user home. The issue was still the same.

Later I ran the below set of command as postgres user

chmod 755 /var/lib/pgsql/.ssh
chmod 600 /var/lib/pgsql/.ssh/authorized_keys
restorecon -R -v /var/lib/pgsql/.ssh

Now the passwordless ssh worked fine.
Now we need to update /etc/barman.conf, which is at the moment owned by root.
So if in case we have sudo setup for barman we can use it or login as root to perform the below operation....


[barman]
compression = gzip
immediate_checkpoint = false
basebackup_retry_times = 3
basebackup_retry_sleep = 30
last_backup_maximum_age = 1 DAYS
[per postgres instance]
description = "Main Postgres1l 15 DB Server"
ssh_command = ssh postgres@127.0.0.1
conninfo = host=127.0.0.1 user=postgres
retention_policy_mode = auto
retention_policy = RECOVERY WINDOW OF 7 days
wal_retention_policy = main

>>>> configuration @ barman side is complete. Final config values are...

[root@10 ~]# grep -Ev ";|^$" /etc/barman.conf
[barman]
barman_user = barman
configuration_files_directory = /etc/barman.d
barman_home = /var/lib/barman
log_file = /var/log/barman/barman.log
log_level = INFO
compression = gzip
immediate_checkpoint = false
basebackup_retry_times = 3
basebackup_retry_sleep = 30
last_backup_maximum_age = 1 DAYS
[postgres15-dbserver]
description = "Main Postgres1l 15 DB Server"
ssh_command = ssh postgres@127.0.0.1
conninfo = host=127.0.0.1 user=postgres
retention_policy_mode = auto
retention_policy = RECOVERY WINDOW OF 7 days
wal_retention_policy = main
[root@10 ~]#

Find out the incoming_wals_directory:
barman show-server postgres15-dbserver | grep incoming_wals_directory

Actual output:

-bash-4.2$ barman show-server postgres15-dbserver | grep incoming_wals_directory
WARNING: No backup strategy set for server 'postgres15-dbserver' (using default 'concurrent_backup').
WARNING: No archiver enabled for server 'postgres15-dbserver'. Please turn on 'archiver', 'streaming_archiver' or both
WARNING: Forcing 'archiver = on'
        incoming_wals_directory: /var/lib/barman/postgres15-dbserver/incoming
-bash-4.2$

Now changed the postgresql.conf with the below setting & restarted postgresql service:

-bash-4.2$ grep -E "wal_level|archive_mode|archive_command" postgresql.conf
wal_level = replica                     # minimal, replica, or logical
archive_mode = on               # enables archiving; off, on, or always
                                # (empty string indicates archive_command should
archive_command = 'rsync -a %p barman@127.0.0.1:/var/lib/barman/postgres15-dbserver/incoming/%f'                # command to use to archive a logfile segment
-bash-4.2$

[root@10 ~]# systemctl restart postgresql-15
[root@10 ~]# systemctl status postgresql-15
● postgresql-15.service - PostgreSQL 15 database server
   Loaded: loaded (/usr/lib/systemd/system/postgresql-15.service; enabled; vendor preset: disabled)
  Drop-In: /etc/systemd/system/postgresql-15.service.d
           └─override.conf
   Active: active (running) since Tue 2022-10-25 22:06:25 IST; 6s ago
     Docs: https://www.postgresql.org/docs/15/static/
  Process: 1985 ExecStartPre=/usr/pgsql-15/bin/postgresql-15-check-db-dir ${PGDATA} (code=exited, status=0/SUCCESS)
 Main PID: 1990 (postmaster)
   CGroup: /system.slice/postgresql-15.service
           ├─1990 /usr/pgsql-15/bin/postmaster -D /pgDATA/data
           ├─1992 postgres: logger
           ├─1993 postgres: checkpointer
           ├─1994 postgres: background writer
           ├─1996 postgres: walwriter
           ├─1997 postgres: autovacuum launcher
           ├─1998 postgres: archiver
           └─1999 postgres: logical replication launcher
Oct 25 22:06:24 10.0.2.4 systemd[1]: Stopped PostgreSQL 15 database server.
Oct 25 22:06:24 10.0.2.4 systemd[1]: Starting PostgreSQL 15 database server...
Oct 25 22:06:25 10.0.2.4 postmaster[1990]: 2022-10-25 22:06:25.210 IST [1990] LOG:  redirecting log output to logg...ocess
Oct 25 22:06:25 10.0.2.4 postmaster[1990]: 2022-10-25 22:06:25.210 IST [1990] HINT:  Future log output will appear...log".
Oct 25 22:06:25 10.0.2.4 systemd[1]: Started PostgreSQL 15 database server.
Hint: Some lines were ellipsized, use -l to show in full.
[root@10 ~]#

BARMAN check:

command: barman check postgres15-dbserver
-bash-4.2$ barman check postgres15-dbserver
WARNING: No backup strategy set for server 'postgres15-dbserver' (using default 'concurrent_backup').
WARNING: No archiver enabled for server 'postgres15-dbserver'. Please turn on 'archiver', 'streaming_archiver' or both
WARNING: Forcing 'archiver = on'
Server postgres15-dbserver:
        WAL archive: FAILED (please make sure WAL shipping is setup) <<<<<<<<<< To work around this error, we have to run the next command.
        PostgreSQL: OK
        superuser or standard user with backup privileges: OK
        wal_level: OK
        directories: OK
        retention policy settings: OK
        backup maximum age: FAILED (interval provided: 1 day, latest backup age: No available backups)
        backup minimum size: OK (0 B)
        wal maximum age: OK (no last_wal_maximum_age provided)
        wal size: OK (0 B)
        compression settings: OK
        failed backups: OK (there are 0 failed backups)
        minimum redundancy requirements: OK (have 0 backups, expected at least 0)
        ssh: OK (PostgreSQL server)
        systemid coherence: OK (no system Id stored on disk)
        archive_mode: OK
        archive_command: OK
        continuous archiving: OK
        archiver errors: OK
-bash-4.2$

I have kept irewalld down for now.

barman switch-wal --force --archive postgres15-dbserver

Actual output:

-bash-4.2$  barman switch-wal --force --archive postgres15-dbserver
WARNING: No backup strategy set for server 'postgres15-dbserver' (using default 'concurrent_backup').
WARNING: No archiver enabled for server 'postgres15-dbserver'. Please turn on 'archiver', 'streaming_archiver' or both
WARNING: Forcing 'archiver = on'
The WAL file 000000010000000100000000 has been closed on server 'postgres15-dbserver'
Waiting for the WAL file 000000010000000100000000 from server 'postgres15-dbserver' (max: 30 seconds)
Processing xlog segments from file archival for postgres15-dbserver
        000000010000000100000000

>>>> the switch-wal finished fine.

-bash-4.2$ barman check postgres15-dbserver
WARNING: No backup strategy set for server 'postgres15-dbserver' (using default 'concurrent_backup').
WARNING: No archiver enabled for server 'postgres15-dbserver'. Please turn on 'archiver', 'streaming_archiver' or both
WARNING: Forcing 'archiver = on'
Server postgres15-dbserver:
        PostgreSQL: OK <<<< No more error or warnings.
        superuser or standard user with backup privileges: OK
        wal_level: OK
        directories: OK
        retention policy settings: OK
        backup maximum age: FAILED (interval provided: 1 day, latest backup age: No available backups)
        backup minimum size: OK (0 B)
        wal maximum age: OK (no last_wal_maximum_age provided)
        wal size: OK (0 B)
        compression settings: OK
        failed backups: OK (there are 0 failed backups)
        minimum redundancy requirements: OK (have 0 backups, expected at least 0)
        ssh: OK (PostgreSQL server)
        systemid coherence: OK (no system Id stored on disk)
        archive_mode: OK
        archive_command: OK
        continuous archiving: OK
        archiver errors: OK
-bash-4.2$

barman list-server

Actual output:

-bash-4.2$ barman list-server
WARNING: No backup strategy set for server 'postgres15-dbserver' (using default 'concurrent_backup').
WARNING: No archiver enabled for server 'postgres15-dbserver'. Please turn on 'archiver', 'streaming_archiver' or both
WARNING: Forcing 'archiver = on'
postgres15-dbserver - Main Postgres1l 15 DB Server
-bash-4.2$

Creating the first backup:

barman backup postgres15-dbserver

Actual output:

-bash-4.2$ barman backup postgres15-dbserver
WARNING: No backup strategy set for server 'postgres15-dbserver' (using default 'concurrent_backup').
WARNING: No archiver enabled for server 'postgres15-dbserver'. Please turn on 'archiver', 'streaming_archiver' or both
WARNING: Forcing 'archiver = on'
Starting backup using rsync-concurrent method for server postgres15-dbserver in /var/lib/barman/postgres15-dbserver/base/20221025T233915
Backup start at LSN: 1/2000028 (000000010000000100000002, 00000028)
This is the first backup for server postgres15-dbserver
WAL segments preceding the current backup have been found:
        0000000100000000000000FD from server postgres15-dbserver has been removed
        0000000100000000000000FE from server postgres15-dbserver has been removed
        0000000100000000000000FF from server postgres15-dbserver has been removed
        000000010000000100000000 from server postgres15-dbserver has been removed
Starting backup copy via rsync/SSH for 20221025T233915
Copy done (time: 1 minute, 52 seconds)
This is the first backup for server postgres15-dbserver
WAL segments preceding the current backup have been found:
        000000010000000100000001 from server postgres15-dbserver has been removed
Asking PostgreSQL server to finalize the backup.
Backup size: 2.1 GiB
Backup end at LSN: 1/2000170 (000000010000000100000002, 00000170)
Backup completed (start time: 2022-10-25 23:39:15.563684, elapsed time: 2 minutes, 1 second)
Processing xlog segments from file archival for postgres15-dbserver
        000000010000000100000002
        000000010000000100000002.00000028.backup
        000000010000000100000003
-bash-4.2$

barman list-backup postgres15-dbserver

Actual output:

-bash-4.2$ barman list-backup postgres15-dbserver
WARNING: No backup strategy set for server 'postgres15-dbserver' (using default 'concurrent_backup').
WARNING: No archiver enabled for server 'postgres15-dbserver'. Please turn on 'archiver', 'streaming_archiver' or both
WARNING: Forcing 'archiver = on'
postgres15-dbserver 20221025T233915 - Tue Oct 25 23:41:11 2022 - Size: 2.1 GiB - WAL Size: 16.0 KiB
-bash-4.2$
barman list-files postgres15-dbserver 20221025T233915
Actual output:
-bash-4.2$ barman list-files postgres15-dbserver 20221025T233915
WARNING: No backup strategy set for server 'postgres15-dbserver' (using default 'concurrent_backup').
WARNING: No archiver enabled for server 'postgres15-dbserver'. Please turn on 'archiver', 'streaming_archiver' or both
WARNING: Forcing 'archiver = on'
/var/lib/barman/postgres15-dbserver/base/20221025T233915/backup.info
/var/lib/barman/postgres15-dbserver/base/20221025T233915/data/PG_VERSION
/var/lib/barman/postgres15-dbserver/base/20221025T233915/data/backup_label
/var/lib/barman/postgres15-dbserver/base/20221025T233915/data/current_logfiles
...
/var/lib/barman/postgres15-dbserver/base/20221025T233915/data/global/6247
/var/lib/barman/postgres15-dbserver/base/20221025T233915/data/global/pg_control
/var/lib/barman/postgres15-dbserver/base/20221025T233915/data/global/pg_filenode.map
/var/lib/barman/postgres15-dbserver/base/20221025T233915/data/pg_logical/replorigin_checkpoint
/var/lib/barman/postgres15-dbserver/base/20221025T233915/data/pg_multixact/members/0000
/var/lib/barman/postgres15-dbserver/base/20221025T233915/data/pg_multixact/offsets/0000
/var/lib/barman/postgres15-dbserver/base/20221025T233915/data/pg_xact/0000
/var/lib/barman/postgres15-dbserver/wals/0000000100000001/000000010000000100000002
-bash-4.2$
After last backup no more FAILED state in check output of barman.
-bash-4.2$ barman check postgres15-dbserver
WARNING: No backup strategy set for server 'postgres15-dbserver' (using default 'concurrent_backup').
WARNING: No archiver enabled for server 'postgres15-dbserver'. Please turn on 'archiver', 'streaming_archiver' or both
WARNING: Forcing 'archiver = on'
Server postgres15-dbserver:
        PostgreSQL: OK
        superuser or standard user with backup privileges: OK
        wal_level: OK
        directories: OK
        retention policy settings: OK
        backup maximum age: OK (interval provided: 1 day, latest backup age: 8 minutes, 34 seconds)
        backup minimum size: OK (2.1 GiB)
        wal maximum age: OK (no last_wal_maximum_age provided)
        wal size: OK (16.0 KiB)
        compression settings: OK
        failed backups: OK (there are 0 failed backups)
        minimum redundancy requirements: OK (have 1 backups, expected at least 0)
        ssh: OK (PostgreSQL server)
        systemid coherence: OK
        archive_mode: OK
        archive_command: OK
        continuous archiving: OK
        archiver errors: OK
-bash-4.2$

This completes barman backup. We have to schedule regular backup in cron of barman.
Thanks

Monday, October 17, 2022

Yum or rpm based postgresql version 15 setup

Objective:

In this blog, we will see how we setup postgresql 15 using yum or rpm.

>>> Ensure the repository configuration file is updated to exclude the postgresql package from base and updates section to avoid later package incorrect resolutions

[root@10 ~]# ls -altr /etc/yum.repos.d/
total 56
-rw-r--r--.  1 root root  616 Nov 23  2020 CentOS-x86_64-kernel.repo
-rw-r--r--.  1 root root 8515 Nov 23  2020 CentOS-Vault.repo
-rw-r--r--.  1 root root 1331 Nov 23  2020 CentOS-Sources.repo
-rw-r--r--.  1 root root  630 Nov 23  2020 CentOS-Media.repo
-rw-r--r--.  1 root root  314 Nov 23  2020 CentOS-fasttrack.repo
-rw-r--r--.  1 root root  649 Nov 23  2020 CentOS-Debuginfo.repo
-rw-r--r--.  1 root root 1309 Nov 23  2020 CentOS-CR.repo
-rw-r--r--.  1 root root 1664 Nov 23  2020 CentOS-Base.repo
-rw-r--r--.  1 root root 1664 Oct 16 16:08 CentOS-Base.repo_bkp_16oct2022
drwxr-xr-x.  2 root root  258 Oct 16 16:08 .
drwxr-xr-x. 74 root root 8192 Oct 16 21:29 ..
[root@10 ~]# 

>>> Edit the CentOS-Base.repo

vi /etc/yum.repos.d/CentOS-Base.repo
[base]
name=CentOS-$releasever - Base
mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=os&infra=$infra
#baseurl=http://mirror.centos.org/centos/$releasever/os/$basearch/
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
exclude=postgresql*
#released updates
[updates]
name=CentOS-$releasever - Updates
mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=updates&infra=$infra
#baseurl=http://mirror.centos.org/centos/$releasever/updates/$basearch/
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
exclude=postgresql*

>>> Goto url
https://www.postgresql.org/download/linux/redhat/
Choose relevant options for the install. Copy the scripts supplied...... [Remember postgresql readme says the binary of the postgresql isnt owned by the postgres user as like oracle, where oracle owns the oracle rdbms binary]

sudo yum install -y https://download.postgresql.org/pub/repos/yum/reporpms/EL-7-x86_64/pgdg-redhat-repo-latest.noarch.rpm
sudo yum install -y postgresql15-server
sudo /usr/pgsql-15/bin/postgresql-15-setup initdb
sudo systemctl enable postgresql-15
sudo systemctl start postgresql-15


Command: yum install -y https://download.postgresql.org/pub/repos/yum/reporpms/EL-7-x86_64/pgdg-redhat-repo-latest.noarch.rpm

Installed file:
-rw-r--r--.  1 root root 10693 Sep 29 19:52 pgdg-redhat-all.repo

I removed sudo, since I ran the commands as root user. Right now I dont have postgres user either, since the postgres user is needed only to own and manage postgres data not its binary

Command: yum install -y postgresql15-server
Failed due to 
...
Error: Package: postgresql15-15.0-1PGDG.rhel7.x86_64 (pgdg15)
           Requires: libzstd >= 1.4.0
Error: Package: postgresql15-server-15.0-1PGDG.rhel7.x86_64 (pgdg15)
           Requires: libzstd.so.1()(64bit)
Error: Package: postgresql15-15.0-1PGDG.rhel7.x86_64 (pgdg15)
           Requires: libzstd.so.1()(64bit)
 You could try using --skip-broken to work around the problem
 You could try running: rpm -Va --nofiles --nodigest


We need epel for this action:
yum -y install epel-release [which has the package libzstd]

yum install libzstd-devel

Install output:


[root@10 yum.repos.d]# yum install libzstd-devel
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * base: mirrors.nxtgen.com
 * epel: download.nus.edu.sg
 * extras: mirrors.nxtgen.com
 * updates: mirrors.nxtgen.com
Resolving Dependencies
--> Running transaction check
---> Package libzstd-devel.x86_64 0:1.5.2-1.el7 will be installed
--> Processing Dependency: libzstd(x86-64) = 1.5.2-1.el7 for package: libzstd-devel-1.5.2-1.el7.x86_64
--> Processing Dependency: libzstd.so.1()(64bit) for package: libzstd-devel-1.5.2-1.el7.x86_64
--> Running transaction check
---> Package libzstd.x86_64 0:1.5.2-1.el7 will be installed
--> Finished Dependency Resolution
Dependencies Resolved
==========================================================================================================================
 Package                          Arch                      Version                         Repository               Size
==========================================================================================================================
Installing:
 libzstd-devel                    x86_64                    1.5.2-1.el7                     epel                     47 k
Installing for dependencies:
 libzstd                          x86_64                    1.5.2-1.el7                     epel                    282 k
Transaction Summary
==========================================================================================================================
Install  1 Package (+1 Dependent package)
Total download size: 330 k
Installed size: 918 k
Is this ok [y/d/N]: y
Downloading packages:
warning: /var/cache/yum/x86_64/7/epel/packages/libzstd-devel-1.5.2-1.el7.x86_64.rpm: Header V4 RSA/SHA256 Signature, key ID 352c64e5: NOKEY
Public key for libzstd-devel-1.5.2-1.el7.x86_64.rpm is not installed
(1/2): libzstd-devel-1.5.2-1.el7.x86_64.rpm                                                        |  47 kB  00:00:06
(2/2): libzstd-1.5.2-1.el7.x86_64.rpm                                                              | 282 kB  00:00:08
--------------------------------------------------------------------------------------------------------------------------
Total                                                                                      37 kB/s | 330 kB  00:00:08
Retrieving key from file:///etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-7
Importing GPG key 0x352C64E5:
 Userid     : "Fedora EPEL (7) <epel@fedoraproject.org>"
 Fingerprint: 91e9 7d7c 4a5e 96f1 7f3e 888f 6a2f aea2 352c 64e5
 Package    : epel-release-7-11.noarch (@extras)
 From       : /etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-7
Is this ok [y/N]: y
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Installing : libzstd-1.5.2-1.el7.x86_64                                                                             1/2
  Installing : libzstd-devel-1.5.2-1.el7.x86_64                                                                       2/2
  Verifying  : libzstd-1.5.2-1.el7.x86_64                                                                             1/2
  Verifying  : libzstd-devel-1.5.2-1.el7.x86_64                                                                       2/2
Installed:
  libzstd-devel.x86_64 0:1.5.2-1.el7
Dependency Installed:
  libzstd.x86_64 0:1.5.2-1.el7
Complete!


[root@10 yum.repos.d]# ls -altr
total 80
-rw-r--r--.  1 root root  1050 Oct  2  2017 epel-testing.repo
-rw-r--r--.  1 root root   951 Oct  2  2017 epel.repo
-rw-r--r--.  1 root root   616 Nov 23  2020 CentOS-x86_64-kernel.repo
-rw-r--r--.  1 root root  8515 Nov 23  2020 CentOS-Vault.repo
-rw-r--r--.  1 root root  1331 Nov 23  2020 CentOS-Sources.repo
-rw-r--r--.  1 root root   630 Nov 23  2020 CentOS-Media.repo
-rw-r--r--.  1 root root   314 Nov 23  2020 CentOS-fasttrack.repo
-rw-r--r--.  1 root root   649 Nov 23  2020 CentOS-Debuginfo.repo
-rw-r--r--.  1 root root  1309 Nov 23  2020 CentOS-CR.repo
-rw-r--r--.  1 root root 10693 Sep 29 19:52 pgdg-redhat-all.repo-new
-rw-r--r--.  1 root root  1664 Oct 16 16:08 CentOS-Base.repo_bkp_16oct2022
-rw-r--r--.  1 root root  1706 Oct 16 22:35 CentOS-Base.repo
drwxr-xr-x.  2 root root  4096 Oct 16 22:35 .
drwxr-xr-x. 74 root root  8192 Oct 16 22:39 ..

[root@10 yum.repos.d]# mv pgdg-redhat-all.repo-new pgdg-redhat-all.repo

postgresql 15 install:

[root@10 yum.repos.d]# yum install -y postgresql15-server
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * base: centos.excellmedia.net
 * epel: download.nus.edu.sg
 * extras: centos.excellmedia.net
 * updates: centos.excellmedia.net
Resolving Dependencies
--> Running transaction check
---> Package postgresql15-server.x86_64 0:15.0-1PGDG.rhel7 will be installed
--> Processing Dependency: postgresql15-libs(x86-64) = 15.0-1PGDG.rhel7 for package: postgresql15-server-15.0-1PGDG.rhel7.x86_64
--> Processing Dependency: postgresql15(x86-64) = 15.0-1PGDG.rhel7 for package: postgresql15-server-15.0-1PGDG.rhel7.x86_64
--> Processing Dependency: libpq.so.5()(64bit) for package: postgresql15-server-15.0-1PGDG.rhel7.x86_64
--> Processing Dependency: libicuuc.so.50()(64bit) for package: postgresql15-server-15.0-1PGDG.rhel7.x86_64
--> Processing Dependency: libicui18n.so.50()(64bit) for package: postgresql15-server-15.0-1PGDG.rhel7.x86_64
--> Running transaction check
---> Package libicu.x86_64 0:50.2-4.el7_7 will be installed
---> Package postgresql15.x86_64 0:15.0-1PGDG.rhel7 will be installed
---> Package postgresql15-libs.x86_64 0:15.0-1PGDG.rhel7 will be installed
--> Finished Dependency Resolution
Dependencies Resolved
==========================================================================================================================
 Package                             Arch                   Version                          Repository              Size
==========================================================================================================================
Installing:
 postgresql15-server                 x86_64                 15.0-1PGDG.rhel7                 pgdg15                 5.6 M
Installing for dependencies:
 libicu                              x86_64                 50.2-4.el7_7                     base                   6.9 M
 postgresql15                        x86_64                 15.0-1PGDG.rhel7                 pgdg15                 1.5 M
 postgresql15-libs                   x86_64                 15.0-1PGDG.rhel7                 pgdg15                 277 k
Transaction Summary
==========================================================================================================================
Install  1 Package (+3 Dependent packages)
Total download size: 14 M
Installed size: 55 M
Downloading packages:
warning: /var/cache/yum/x86_64/7/pgdg15/packages/postgresql15-libs-15.0-1PGDG.rhel7.x86_64.rpm: Header V4 DSA/SHA1 Signature, key ID 442df0f8: NOKEY
Public key for postgresql15-libs-15.0-1PGDG.rhel7.x86_64.rpm is not installed
(1/4): postgresql15-libs-15.0-1PGDG.rhel7.x86_64.rpm                                               | 277 kB  00:00:10
(2/4): libicu-50.2-4.el7_7.x86_64.rpm                                                              | 6.9 MB  00:00:27
(3/4): postgresql15-15.0-1PGDG.rhel7.x86_64.rpm                                                    | 1.5 MB  00:00:29
(4/4): postgresql15-server-15.0-1PGDG.rhel7.x86_64.rpm                                             | 5.6 MB  00:00:40
--------------------------------------------------------------------------------------------------------------------------
Total                                                                                     286 kB/s |  14 MB  00:00:51
Retrieving key from file:///etc/pki/rpm-gpg/RPM-GPG-KEY-PGDG
Importing GPG key 0x442DF0F8:
 Userid     : "PostgreSQL RPM Building Project <pgsql-pkg-yum@postgresql.org>"
 Fingerprint: 68c9 e2b9 1a37 d136 fe74 d176 1f16 d2e1 442d f0f8
 Package    : pgdg-redhat-repo-42.0-27.noarch (@/pgdg-redhat-repo-latest.noarch)
 From       : /etc/pki/rpm-gpg/RPM-GPG-KEY-PGDG
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Installing : postgresql15-libs-15.0-1PGDG.rhel7.x86_64                                                              1/4
  Installing : libicu-50.2-4.el7_7.x86_64                                                                             2/4
  Installing : postgresql15-15.0-1PGDG.rhel7.x86_64                                                                   3/4
  Installing : postgresql15-server-15.0-1PGDG.rhel7.x86_64                                                            4/4
  Verifying  : postgresql15-server-15.0-1PGDG.rhel7.x86_64                                                            1/4
  Verifying  : libicu-50.2-4.el7_7.x86_64                                                                             2/4
  Verifying  : postgresql15-libs-15.0-1PGDG.rhel7.x86_64                                                              3/4
  Verifying  : postgresql15-15.0-1PGDG.rhel7.x86_64                                                                   4/4
Installed:
  postgresql15-server.x86_64 0:15.0-1PGDG.rhel7
Dependency Installed:
  libicu.x86_64 0:50.2-4.el7_7    postgresql15.x86_64 0:15.0-1PGDG.rhel7    postgresql15-libs.x86_64 0:15.0-1PGDG.rhel7
Complete!
[root@10 yum.repos.d]#

>>> user and group postgres is by default added in the course...

[root@10 yum.repos.d]# groupadd postgres
groupadd: group 'postgres' already exists

[root@10 yum.repos.d]# id postgres
uid=26(postgres) gid=26(postgres) groups=26(postgres)
[root@10 yum.repos.d]#

>>> Initalizing the data directory:

Ex.:
root# mkdir /usr/local/pgsql
root# chown postgres /usr/local/pgsql
root# su postgres
postgres$ initdb -D /usr/local/pgsql/data

Actual: <<< we use the data directory we seperately created.
mkdir /pgDATA/data
[root@10 yum.repos.d]# chown postgres:postgres /pgDATA/data
[root@10 yum.repos.d]# ls -ld /pgDATA/data
drwxr-xr-x. 2 postgres postgres 6 Oct 16 22:59 /pgDATA/data
[root@10 yum.repos.d]#

>>> Update the .bash_profile of postgres user:

-bash-4.2$ cat .bash_profile
[ -f /etc/profile ] && source /etc/profile
PGDATA=/pgDATA/data  << appended
export PGDATA
# If you want to customize your settings,
# Use the file below. This is not overridden
# by the RPMS.
[ -f /var/lib/pgsql/.pgsql_profile ] && source /var/lib/pgsql/.pgsql_profile
export PATH=$PATH:/usr/pgsql-15/bin:/usr/pgsql-15/lib:  <<< appended
-bash-4.2$

>>> logged in as postgres user and initialized the db

initdb -D /pgDATA/data

-bash-4.2$ echo $PGDATA
/pgDATA/data

-bash-4.2$ initdb -D $PGDATA
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.

The database cluster will be initialized with locale "en_US.UTF-8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".

Data page checksums are disabled.

fixing permissions on existing directory /pgDATA/data ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Asia/Kolkata
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok

initdb: warning: enabling "trust" authentication for local connections
initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.

Success. You can now start the database server using:

    pg_ctl -D /pgDATA/data -l logfile start

-bash-4.2$

The cluster database is initialized.

OKV platform certificate rotation - pitfall , awareness!!!!

OKV version: 21.9 Setup: Multimaster R/W cluster Plan: https://docs.google.com/spreadsheets/d/e/2PACX-1vSaXXTjj9cE1fvYpNmsDNBOkTIw78yTwQ6a9o...