May 6, 2013

Redhat Enterprise Linux patching: Rollback strategies

Posted in Linux at 10:25 am by alessiodini


Soon i will patch a lot of systems ( RHEL 5 – RHEL 6 ) and i’m planning every rollback strategy.
Studying and looking for the web I found that people usually use RPM rollback but this is NOT RECOMMENDED!!

People on RHEL 5 add RPM macro , on RHEL 6 use yum history but both struments can be used for small rollbacks, not for minor release rollbacks!

I read this from Redhat:

Note: Downgrading a system to minor version (ex: RHEL-6.1 to RHEL-6.0) is not recommended as this might leave the system in broken state where libgcc and other libraries won’t rollback as expected. Use the history option for small update rollbacks.

Same is for RHEL 5.

So what can I think about rollback?

1) LVM snapshot ( usually estimated as ~ 20% lvol source space )
2) Files copy from / and /boot partitions with dd. This is useful with soft/hard links and it’s bit a bit copy. I just need a filesystem from SAN or NFS. If snapshot will not work I just need to boot the system in rescue mode , configure the network , mount the filesystems and use dd for rollback!

April 2, 2013

Oracle 11g installation on ESXi/SLES 11 packages issue

Posted in Oracle at 2:24 pm by alessiodini


In these days i’m working on a ESXi/SLES 11 Cluster/SAP infrastructure.

When we were installing Oracle , we saw that some package was missing … I expected this.
Looking for these packages I did not found any of them and I was really surprised about that.
I looked on the web and I found that this is a known issue.
When Oracle installation begins , on a physical system it correctly determines the operating system ( in that case SLES 11 ). In ESXi/virtual environment something goes wrong and it treats the system as Oracle Enterprise Linux ( bug ? ).
That’s why i was not able to look for that packages! They are for OEL , not for SLES :)

We solved this issue changing a parameter in a configuration file.
In our case we were installing database , so we had:

/oracle/stage/112_64/database/stage/cvu/cv/admin/cvu_config

we changed CV_ASSUME_DISTID=OEL4 in CV_ASSUME_DISTID=SUSE11

March 20, 2013

SLES 11 SP2: Ext4 readonly mount

Posted in Linux at 10:30 am by alessiodini


Yesterday i was doing some test on my SLES 11 for SAP.
I was trying to mount an ext4 filesystem on /mnt.

During the mount i saw the message:

mount: warning: /mnt seems to be mounted read-only

I was surprised to see that message , so i tought about some problem with lvm or iSCSi ( i maked a volume group on a disk via iSCSi ).

Looking the system messages i saw this:

Mar 19 15:59:19 suse1 kernel: [ 1147.951164] EXT4-fs (dm-2): ext4 is supported in read-only mode only

Now it was clear! I understand too why SLESS 11 had the root filesystem on ext3 :)

February 26, 2013

Linux grub Error 1: Cannot mount selected partition

Posted in Linux at 11:54 am by alessiodini


In these days i’m playing a lot with mdadm and grub on Linux. I need to learn how make a root disks mirror starting from /dev/sd* booting devices.

During the tasks i did, I encoutered a problem with grub. I had:

- hd0 (sda)
- hd1 (sdb)

/dev/md1 -> /dev/sdb1 ( /boot )
/dev/md3 -> /dev/sdb3 ( / )
/dev/md2 -> /dev/sdb2 ( swap )

I edited /boot/grub/grub.conf as follow:

# grub.conf generated by anaconda
#
# Note that you do not have to rerun grub after making changes to this file
# NOTICE: You have a /boot partition. This means that
# all kernel and initrd paths are relative to /boot/, eg.
# root (hd0,0)
# kernel /vmlinuz-version ro root=/dev/sda3
# initrd /initrd-[generic-]version.img
#boot=/dev/sda
default=0
fallback=1
timeout=5
splashimage=(hd0,0)/grub/splash.xpm.gz
hiddenmenu
title CentOS MD (2.6.32-279.el6.x86_64)
root (hd1,0)
kernel /vmlinuz-2.6.32-279.el6.x86_64 ro root=/dev/md3 rd_NO_LUKS LANG=en_US.UTF-8 KEYBOARDTYPE=pc KEYTABLE=it rd_NO_MD SYSFONT=latarcyrheb-sun16 crashkernel=auto rd_NO_LVM rd_NO_DM rhgb quiet
initrd /initramfs-2.6.32-279.el6.x86_64.img
title CentOS (2.6.32-279.el6.x86_64)
root (hd0,0)
kernel /vmlinuz-2.6.32-279.el6.x86_64 ro root=UUID=7c1ba3b1-756c-4124-9079-d11a32ceea78 rd_NO_LUKS LANG=en_US.UTF-8 KEYBOARDTYPE=pc KEYTABLE=it rd_NO_MD SYSFONT=latarcyrheb-sun16 crashkernel=auto rd_NO_LVM rd_NO_DM rhgb quiet
initrd /initramfs-2.6.32-279.el6.x86_64.img

After i tried to run:
[root@localhost grub]# grub
Probing devices to guess BIOS drives. This may take a long time.

GNU GRUB version 0.97 (640K lower / 3072K upper memory)

[ Minimal BASH-like line editing is supported. For the first word, TAB
lists possible command completions. Anywhere else TAB lists the possible
completions of a device/filename.]
grub> root (hd0,0)
root (hd0,0)
Filesystem type is ext2fs, partition type 0×83
grub> setup (hd0)
setup (hd0)
Checking if “/boot/grub/stage1″ exists… no
Checking if “/grub/stage1″ exists… yes
Checking if “/grub/stage2″ exists… yes
Checking if “/grub/e2fs_stage1_5″ exists… yes
Running “embed /grub/e2fs_stage1_5 (hd0)”… 27 sectors are embedded.
succeeded
Running “install /grub/stage1 (hd0) (hd0)1+27 p (hd0,0)/grub/stage2 /grub/grub.conf”… succeeded
Done.
grub> root (hd1,0)
root (hd1,0)
Filesystem type unknown, partition type 0xfd
grub> setup (hd1)
setup (hd1)

Error 1: Cannot mount selected partition
grub>

Looking on the web i found that the problem is with md devices metadata.
I maked them with:
mdadm –create /dev/md1 –level=1 –raid-disks=2 –metadata=default missing /dev/sdb1

default metadata is 1.2 , and it’s incompatible with GRUB!
So for boot disks the solution is make them with:
mdadm –create /dev/md1 –level=1 –raid-disks=2 –metadata=0.90 missing /dev/sdb1

February 18, 2013

Linux CentOS 6 e Windows 7 dual boot – BOOTMGR is missing

Posted in Linux at 1:38 pm by alessiodini


Today I installed CentOS6 on my notebook for KVM purposes.
After the installation I tried to test Windows boot and i got this error:

BOOTMGR is missing
Press CTRL + ALT + DEL to restart

I immediately tought “damn i’m using my work-notebook this is bad”.
From Linux I saw /etc/grub entries and for Windows I saw:

root (hd0,7)

Watching fdisk -l output i saw the first partition with little size and I tried to map it as follow:

root (hd0,1)

After this I was able to boot Windows :)

February 8, 2013

Linux CentOS 6: strange behaviour with multiple devices on single mount point !!!

Posted in Linux at 11:18 am by alessiodini


I’m doing a lot of tests , experiments on my CentOS.

Yesterday i was experimenti a LVM mirror recovery and i found two devices mounted on same mount point: /mnt

I tought “damn due to my experiments i erased the system again…”

Later i was curious about to try it again. IT WORKED , how is that possible?

I can do this even now:

[root@centos6 ~]# cat /etc/issue
CentOS release 6.3 (Final)
Kernel \r on an \m

[root@centos6 ~]# lvs
LV VG Attr LSize Pool Origin Data% Move Log Copy% Convert
ale rootvg -wi-a— 1.00g
ale2 rootvg -wi-a— 104.00m
mau rootvg -wi-a— 1.00g
rootvol rootvg -wi-ao– 13.46g
swap2vol rootvg -wi-a— 2.00g
swapvol rootvg -wi-ao– 2.05g
test rootvg -wi-ao– 200.00m
mirrvol testvg mwi-a-m- 2.00g mirrvol_mlog 100.00

[root@centos6 ~]# vgs
VG #PV #LV #SN Attr VSize VFree
rootvg 1 7 0 wz–n- 24.51g 4.70g
testvg 2 1 0 wz–n- 9.98g 5.98g

[root@centos6 ~]# mount /dev/mapper/rootvg-test /mnt
[root@centos6 ~]# mount /dev/mapper/testvg-mirrvol /mnt

[root@centos6 ~]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/rootvg-rootvol
14G 4.4G 8.2G 35% /
tmpfs 498M 188K 498M 1% /dev/shm
/dev/sda1 485M 33M 427M 8% /boot
/dev/mapper/rootvg-test
2.0G 67M 1.9G 4% /mnt

/dev/mapper/testvg-mirrvol
2.0G 67M 1.9G 4% /mnt

I need to understand. NOW !!

January 31, 2013

Redhat 6: how to make a snapshot of root logical volume and rollback it

Posted in Linux at 2:20 pm by alessiodini


RHEL6 introduces new features about LVM. One of these is the snapshots.

I wanted to test it for rollback purpose. I will think about it for a patching task ;)

For this test I maked a screenshot of root logical volume , I deleted /etc directory and restored it from the sreenshot. All worked !

[root@localhost ~]# df -h
 Filesystem Size Used Avail Use% Mounted on
 /dev/mapper/rootvg-rootvol
 23G 3.1G 18G 15% /
 tmpfs 498M 0 498M 0% /dev/shm
 /dev/sda1 485M 33M 427M 8% /boot

[root@localhost ~]# vgs
VG #PV #LV #SN Attr VSize VFree
rootvg 1 2 0 wz–n- 24.51g 0

I saw no free space on rootvg so i added a new disk

[root@localhost ~]# sfdisk -l /dev/sdb

Disk /dev/sdb: 652 cylinders, 255 heads, 63 sectors/track
Units = cylinders of 8225280 bytes, blocks of 1024 bytes, counting from 0

Device Boot Start     End   #cyls    #blocks   Id  System
/dev/sdb1          0+    651     652-   5237158+  8e  Linux LVM
/dev/sdb2          0       -       0          0    0  Empty
/dev/sdb3          0       -       0          0    0  Empty
/dev/sdb4          0       -       0          0    0  Empty

[root@localhost ~]# pvcreate /dev/sdb1
Writing physical volume data to disk “/dev/sdb1″
Physical volume “/dev/sdb1″ successfully created

[root@localhost ~]# vgextend rootvg /dev/sdb1
Volume group “rootvg” successfully extended

[root@localhost ~]# vgs
VG     #PV #LV #SN Attr   VSize  VFree
rootvg   2   2   0 wz–n- 29.50g 4.99g

Nice. After i maked a screenshot with all free space on the vg.

[root@localhost ~]# lvcreate -l 100%FREE –snapshot –name rootsnap /dev/rootvg/rootvol
Logical volume “rootsnap” created
[root@localhost ~]# lvs
LV       VG     Attr     LSize  Pool Origin  Data%  Move Log Copy%  Convert
rootsnap rootvg swi-a-s-  4.99g      rootvol   0.00
rootvol  rootvg owi-aos- 22.46g
swapvol  rootvg -wi-ao–  2.05g

s attribute said to me that rootsnap is a snapshot !

[root@localhost ~]# lvdisplay /dev/rootvg/rootvol
— Logical volume —
LV Path                /dev/rootvg/rootvol
LV Name                rootvol
VG Name                rootvg
LV UUID                MgWa8V-yUnK-hqC0-Kv6X-strT-eccc-Kiwt8S
LV Write Access        read/write
LV Creation host, time localhost.localdomain, 2013-01-29 19:04:19 +0100
LV snapshot status     source of
rootsnap [active]

LV Status              available
# open                 1
LV Size                22.46 GiB
Current LE             5750
Segments               1
Allocation             inherit
Read ahead sectors     auto
- currently set to     256
Block device           253:1

First test I did:

[root@localhost ~]# dd if=/dev/zero of=test.img bs=1024k count=1000
1000+0 records in
1000+0 records out
1048576000 bytes (1.0 GB) copied, 80.6583 s, 13.0 MB/s

THE SPACE ON THE SNAPSHOT MUST BE CHECKED !!!

[root@localhost ~]# lvs
LV       VG     Attr     LSize  Pool Origin  Data%  Move Log Copy%  Convert
rootsnap rootvg swi-a-s-  4.99g      rootvol  18.21
rootvol  rootvg owi-aos- 22.46g
swapvol  rootvg -wi-ao–  2.05g

[root@localhost ~]# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/rootvg-rootvol
23G  4.2G   17G  20% /
tmpfs                 498M     0  498M   0% /dev/shm
/dev/sda1             485M   33M  427M   8% /boot

After i erased /etc directory!

[root@localhost ~]# rm -rf /etc
[root@localhost ~]# cd /etc
-bash: cd: /etc: No such file or directory
[root@localhost ~]# df -h
df: cannot read table of mounted file systems: No such file or directory

I did a reboot and as expected the system was not able to boot up.
So , I used rescue mode from CD and I ran on the shell:

lvconvert –merge rootvg/rootsnap

At next boot the system came up without any problem/warning!

[root@localhost ~]# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/rootvg-rootvol
23G  3.1G   18G  15% /
tmpfs                 498M     0  498M   0% /dev/shm
/dev/sda1             485M   33M  427M   8% /boot

[root@localhost ~]# cd /root
[root@localhost ~]# ls
anaconda-ks.cfg  install.log  install.log.syslog

( The 1 gb file does not exist anymore )

[root@localhost ~]# lvs
LV      VG     Attr     LSize  Pool Origin Data%  Move Log Copy%  Convert
rootvol rootvg -wi-ao– 22.46g
swapvol rootvg -wi-ao–  2.05g

( the snapshot is not present )
With this i saw that the system was rollbacked with success.
So i will use this method on RHEL6 for rollingback after a patching!

January 22, 2013

Redhat 6: how to know if a logical volume is striped, linear or mirrored

Posted in Linux at 3:50 pm by alessiodini


My company said me that I will work soon with Linux systems , so i will do tasks on Solari/Linux :)
I left HP-UX studying for Linux Redhat 6.

For this purpose i’m studying a lot of LVM. Today i played with linear and striped logical volumes. After I had a question: “How i can understand if a volume is linear , striped or mirrored?”

I initially looked for lvdisplay man page , without success. Discussing this with my friend Maurizio Pagani he said me the best command:

[root@centos ~]# lvs -a -o segtype,devices,lv_name,vg_name
Type Devices LV VG
linear /dev/sda2(0) rootvol rootvg
linear /dev/sda2(5825) swapvol rootvg
linear /dev/sdc1(128) alevol testvg
striped /dev/sdc1(0),/dev/sdb2(0) testvol testvg

Cool!

December 13, 2012

Sun Cluster 3.2: resources migration between two clusters

Posted in Sun Cluster at 11:28 am by alessiodini


Last week i was envolved in this task.
Instead of classic scsnapshot method i used xml configuration files for the migration!

I wrote a new document in my library, you can see it here

I suggest to use xml for every migration ;)

December 3, 2012

Redhat 5.5 and Redhat Cluster Suite task!

Posted in Linux at 3:19 pm by alessiodini


The last week i was envolved in a nice task!
I had to install , and configure :

- four DELL servers ( 2x DELL Poweredge R720 and 2x DELL Poweredge R320 ) with Redhat Enterprise 5.5
- two DELL FC Storage ( MD3600f 8GB )

After long time i worked with Linux , it was funny ;)

I installed and used Redhat Cluster Suite too. I encountered many issues but i was funny when i was plying with clustered vgs ( never saw them before )

Following some command i did for clustered vgs

[root@system1 ~]# for i in mpath2p1 mpath9p1 mpath8p1 mpath7p1 mpath6p1;
> do
> pvcreate /dev/mapper/$i
> done
Physical volume “/dev/mapper/mpath2p1″ successfully created
Physical volume “/dev/mapper/mpath9p1″ successfully created
Physical volume “/dev/mapper/mpath8p1″ successfully created
Physical volume “/dev/mapper/mpath7p1″ successfully created
Physical volume “/dev/mapper/mpath6p1″ successfully created

[root@system1 ~]# vgcreate -c y tibcovg /dev/mapper/mpath2p1 /dev/mapper/mpath9p1 /dev/mapper/mpath8p1 /dev/mapper/mpath7p1 /dev/mapper/mpath6p1
Clustered volume group “tibcovg” successfully created

[root@system2]# vgscan
Reading all physical volumes. This may take a while…
Found volume group “tibcovg” using metadata type lvm2
Found volume group “VolGroup00″ using metadata type lvm2

[root@system1 ~]# lvcreate -L 1.2T -n tibcovol tibcovg
Rounding up size to full physical extent 1.20 TB
Logical volume “tibcovol” created

( on second node i ran lvscan )

[root@system1 ~]# vgs
VG #PV #LV #SN Attr VSize VFree
VolGroup00 1 2 0 wz–n- 1.09T 0
tibcovg 5 2 0 wz–nc 1.46T 21.16G

[root@system2 ]# vgs
VG #PV #LV #SN Attr VSize VFree
VolGroup00 1 2 0 wz–n- 1.09T 0
tibcovg 5 2 0 wz–nc 1.46T 21.16G

[root@system1 ~]# mkdir /tibco
[root@system2 mapper]# mkdir /tibco

[root@system1 ~]# mkfs.gfs2 -p lock_dlm -t soa-cluster:tibco -j2 /dev/tibcovg/tibcovol
This will destroy any data on /dev/tibcovg/tibcovol.
It appears to contain a gfs2 filesystem.

Are you sure you want to proceed? [y/n] y
Device: /dev/tibcovg/tibcovol
Blocksize: 4096
Device Size 1228.80 GB (322122752 blocks)
Filesystem Size: 1228.80 GB (322122752 blocks)
Journals: 2
Resource Groups: 4916
Locking Protocol: “lock_dlm”
Lock Table: “soa-cluster:/tibco”
UUID: 5D884F81-39A4-FF56-326B-AC4735620175

On both nodes i did:
[root@system1 ~]# mount /dev/tibcovg/tibcovol /tibco

After i encountered hard problems with device fencing… due to it , during a reboot of a first node , the second node had problems on /tibco fs.

I called some friend for help anyway i learned a lot of things about Redhat Cluster , this was the first time i configured it !

Next page

Follow

Get every new post delivered to your Inbox.

Join 31 other followers