Saturday 31 August 2013

ELA_03_Unix_File_System

Linux Directory Structure (File System Structure) Explained with Examples

Have you wondered why certain programs are located under /bin, or /sbin, or /usr/bin, or /usr/sbin?
For example, less command is located under /usr/bin directory. Why not /bin, or /sbin, or /usr/sbin? What is the different between all these directories?
In this article, let us review the Linux filesystem structures and understand the meaning of individual high-level directories.

1. / – Root

Every single file and directory starts from the root directory.
Only root user has write privilege under this directory.
Please note that /root is root user’s home directory, which is not same as /.

2. /bin – User Binaries

Contains binary executables.
Common linux commands you need to use in single-user modes are located under this directory.
Commands used by all the users of the system are located here.
For example: ps, ls, ping, grep, cp.

3. /sbin – System Binaries

Just like /bin, /sbin also contains binary executables.
But, the linux commands located under this directory are used typically by system aministrator, for system maintenance purpose.
For example: iptables, reboot, fdisk, ifconfig, swapon

4. /etc – Configuration Files

Contains configuration files required by all programs.
This also contains startup and shutdown shell scripts used to start/stop individual programs.
For example: /etc/resolv.conf, /etc/logrotate.conf

5. /dev – Device Files

Contains device files.
These include terminal devices, usb, or any device attached to the system.
For example: /dev/tty1, /dev/usbmon0

6. /proc – Process Information

Contains information about system process.
This is a pseudo filesystem contains information about running process. For example: /proc/{pid} directory contains information about the process with that particular pid.
This is a virtual filesystem with text information about system resources. For example: /proc/uptime

7. /var – Variable Files

var stands for variable files.
Content of the files that are expected to grow can be found under this directory.
This includes — system log files (/var/log); packages and database files (/var/lib); emails (/var/mail); print queues (/var/spool); lock files (/var/lock); temp files needed across reboots (/var/tmp);

8. /tmp – Temporary Files

Directory that contains temporary files created by system and users.
Files under this directory are deleted when system is rebooted.

9. /usr – User Programs

Contains binaries, libraries, documentation, and source-code for second level programs.
/usr/bin contains binary files for user programs. If you can’t find a user binary under /bin, look under /usr/bin. For example: at, awk, cc, less, scp
/usr/sbin contains binary files for system administrators. If you can’t find a system binary under /sbin, look under /usr/sbin. For example: atd, cron, sshd, useradd, userdel
/usr/lib contains libraries for /usr/bin and /usr/sbin
/usr/local contains users programs that you install from source. For example, when you install apache from source, it goes under /usr/local/apache2

10. /home – Home Directories

Home directories for all users to store their personal files.
For example: /home/john, /home/nikita

11. /boot – Boot Loader Files

Contains boot loader related files.
Kernel initrd, vmlinux, grub files are located under /boot
For example: initrd.img-2.6.32-24-generic, vmlinuz-2.6.32-24-generic

12. /lib – System Libraries

Contains library files that supports the binaries located under /bin and /sbin
Library filenames are either ld* or lib*.so.*
For example: ld-2.11.1.so, libncurses.so.5.7

13. /opt – Optional add-on Applications

opt stands for optional.
Contains add-on applications from individual vendors.
add-on applications should be installed under either /opt/ or /opt/ sub-directory.

14. /mnt – Mount Directory

Temporary mount directory where sysadmins can mount filesystems.

15. /media – Removable Media Devices

Temporary mount directory for removable devices.
For examples, /media/cdrom for CD-ROM; /media/floppy for floppy drives; /media/cdrecorder for CD writer

16. /srv – Service Data

srv stands for service.
Contains server specific services related data.
For example, /srv/cvs contains CVS related data.

Enterprise Linux Administration (ELA_00)

Contents:

1. History of Unix and Linux

2. Features

3. Unix File System

4. Installation

5. Rescue and Single User Modes

6. INIT Upstart

7. Basic Skills

8. Compression Utilities

9. Checksums Tools

10. Grep Usage

11. AWK Usage

12. SED Usage

13. Perl Usage

14. User Management

15. File Permissions

16. Access Control Lists

17. Managing File Links

18. Quota Management

19. EXT4 Parted

20. SWAP Configuration

21. Logical Volume Management (LVM)

21.1. LVM Configuration

21.2. LVM Resize

22. Redundant Array of Inexpensive Disks (RAID)

22.1. RAID Introduction

22.2. RAID Configuration

22.3. RAID Management

23. Package Management with RPM

24. Package Management with YUM

25. Cron

26. At batch

27. Syslog Configuration

28. Common Network Utilities

29. Ipv4 Configuration

30. VSFTPD (Very Secure File Transfer Protocol Daemon) Configuration

31. LFTP

32. Service Management

33. DHCPD (Dynamic Host Configuration Protocol Daemon) Configuration

34. BIND (Berkely Internet Naming Daemon) – DNS (Domain Naming Service) Config.

35. Samba Share

36. Network File System (NFS)

37. Secure Shell (SSH)

38. Network Time Protocol (NTP)

39. Linux Proxy

40. Linux MAIL

41. Extended Internet Services Daemon (xinetd)

ELA_01_History_of_Unix_&_Linux

History of Unix and Linux

YEAR	Event
1957	Bell Labs found they needed an operating system for their computer center that at the time was running various batch jobs. The BESYS operating system was created at Bell Labs to deal with these needs.
1965	Bell Labs was adopting third generation computer equipment and decided to join forces with General Electric and MIT to create Multics (Multiplexed Information and Computing Service).
1969	By April 1969, AT&T made a decision to withdraw Multics and go with GECOS. When Multics was withdrawn Ken Thompson and Dennis Ritchie needed to rewrite an operating system in order to play space travel on another smaller machine (a DEC PDP-7 [Programmed Data Processor 4K memory for user programs). The result was a system that a punning colleague called UNICS (UNiplexed Information and Computing Service)--an 'emasculated Multics'.
1969	Summer 1969 Unix was developed.
1969	Linus Torvalds is born.
1971	First edition of Unix released 11/03/1971. The first edition of the "Unix PROGRAMMER'S MANUAL [by] K. Thompson [and] D. M. Ritchie." It includes over 60 commands like: b (compile B program); boot (reboot system); cat (concatenate files); chdir (change working directory); chmod (change access mode); chown (change owner); cp (copy file);ls (list directory contents); mv (move or rename file); roff (run off text); wc (get word count); who (who is one the system). The main thing missing was pipes.
1972	Second edition of Unix released December 06, 1972.
1972	Ritchie rewrote B and called the new language C.
1973	Unix had been installed on 16 sites (all within AT&T/Western Electric); it was publically unveiled at a conference in October.
1973	Third edition of Unix released February 1973
1973	Forth edition of Unix released November 1973
1974	Fifth edition of Unix released June 1974
1974	Thompson went to UC Berkeley to teach for a year, Bill Joy arrived as a new graduate student. Frustrated with ed, Joy developed a more featured editor em.
1975	Sixth edition of Unix released May 1975
1975	Bourne shell is introduced begins being added onto.
1977	1BSD released late 1977
1978	2BSD released mid 1978
1979	Seventh edition of Unix released January 1979
1979	3BSD released late 1979
1979	SCO founded by Doug and Larry Michels as Unix porting and consulting company.
1980	4.0BSD released October 1980
1982	SGI introduces IRIX.
1983	SCO delivers its first packaged Unix system called SCO XENIX System V for Intel 8086 and 8088 processor-based PCs.
1983	The GNU operating system is first announced by Richard Stallman September 27, 1983.
1984	Ultrix 1.0 was released.
1985	Eighth edition of Unix released February 1985
1985	The GNU manifesto is published in the March 1985 issue of Dr. Dobb's Journal. The GNU project starts a year and a half later.
1986	HP-UX 1.0 released.
1986	Ninth edition of Unix released September 1986
1987	Sun and AT&T lay the groundwork for business computing in the next decade with an alliance to develop Unix System V Release 4.
1988	HP-UX 2.0 released.
1988	HP-UX 3.0 released.
1989	SCO ships SCO Unix System V/386, the first volume commercial product licensed by AT&T to use the Unix System trademark.
1989	HP-UX 7.0 released.
1989	Tenth edition of Unix released October 1989
1990	AIX short for Advanced Interactive eXecutive was first entered into the market by IBMFebruary 1990.
1991	Sun unveils Solaris 2 operating environment, specially tuned for symetric multiprocessing.
1991	Linux is introduced by Linus Torvalds, a student in Finland.
1991	HP-UX 8.0 released.
1991	BSD/386 ALPHA First code released to people outside BSDI 12/xx/1991
1992	HP-UX 9.0 released.
1993	NetBSD 0.8 released 04/20/1993
1993	FreeBSD 1.0 released December of 1993
1994	Red Hat Linux is introduced.
1994	Caldera, Inc was founded in 1994 by Ransom Love and Bryan Sparks.
1994	NetBSD 1.0 released 10/26/1994
1995	FreeBSD 2.0 released 01/xx/1995
1995	SCO acquires Unix Systems source technology business from Novell Corporation (which had acquired it from AT&T's Unix System Laboratories). SCO also acquires UnixWare 2 operating system from Novell.
1995	HP-UX 10.0 released.
1995	4.4 BSD Lite Release 2 the true final distribution from the CSRG 06/xx/1995
1996	KDE is started to be developed by Matthias Ettrich
1997	HP-UX 11.0 released.
1997	Caldera ships OpenLinux Standard 1.1 May 5, 1997, the second offering in Caldera's OpenLinux product line
1998	IRIX 6.5 the fifth generation of SGI Unix is released July 6, 1998.
1998	SCO delivers UnixWare 7 operating system.
1998	Sun Solaris 7 operating system released.
1998	FreeBSD 3.0 released 10/16/1998
2000	FreeBSD 4.0 released 03/13/2000
2000	Caldera Systems Inc. announces that Caldera Systems has entered into agreement to acquire the SCO Server Software Division and the Professional Services Division.
2001	Linus Torvalds releases version 2.4 of the Linux Kernel source code on January 4th.
2001	Microsoft files a trademark suit against Lindows.com in December.
2004	Lindows changes it's name to Linspire April 14, 2004.

The first release of Ubuntu is released October 20, 2004.

Saturday 24 August 2013

Bash Scripting

BASH Scripting

Tests

We can perform testing on Strings, Files an integers

#which test ---> To know where it has been installed.

Integer testing:

In Unix 0 Good

1 Bad

In Boolean 0 False

1 True

# echo $? ---> To see previous command's excution status.

-eq == ---> Equal to

-ne != ---> Not equal to

-le <= ---> Less than or equal to

-ge >= ---> Greater than or equal to

-lt < ---> Less than

-gt > ---> Greater than

-nt ---> Newer than

-ot ---> Older than

-e ---> Exists

Commands:

# tac ---> Inverse command of cat

Script 1. Bash script for translating Upper case filenames to lowercase.
GitHub Link

Script 2. Bash script for translating Upper case filenames to lowercase and rename a file.
GitHub Link

Command Chaining:

We can give number of commands at a time in a single line which are separated by";".

In this way we can execute commands one after one serially.

Ex:

# clear ; pwd ; echo hello world

But in this way every command works independently. i.e. irrespective of the previous commands status.

If we want to use a command depending on the previous command's exit status we can use in the below way.

Ex:

# ls -l && pwd

If the exit status of the first command is 0 the second command will run otherwise it won't.

# ls -z || pwd

If the exit status of the first command is 1 the second command will run because in unix ls command doesn't have anargument z so the first command is false.

&& ---> AND

|| ---> OR

# echo "Hello world"

Hello world

# echo "hello world $value"

hello world

# echo 'hello world $value'

hello world $value

Script 3: Interactive script
GitHub Link

Functions:

Like "real" programming languages, Bash has functions, though in a somewhat limited implementation. A function is a subroutine, a code block that implements a set of operations, a "black box" that performs a specified task. Wherever there is repetitive code, when a task repeats with only slight variations in procedure, then consider using a function.

function function_name {

command...
}

or

function_name () {
command...
}

This second form will cheer the hearts of C programmers (and is more portable).

As in C, the function's opening bracket may optionally appear on the second line.

function_name ()
{
command...
}

Script 4: Functions
GitHub Link

Loops

A loop is a block of code that iterates a list of commands as long as the loop control condition is true.

For Loops :

for argument in [ list ]

do action item

done

Script 5: For Loop 1
GitHub Link

Script 6: For Loop 2
GitHub Link

While Loops:
while condition

do action item

done

Script 7: While Loop
GitHub Link

Until Loops:
until condition

do action item

done

Script 8: Until Loop 1

GitHub Link

Script 9: Until Loop 2
GitHub Link

Control Structure:

As may be expected, the shell language contains many special features for working with filenames. It is thus very easy to create control structures based on the simple examples above that perform complex operations on files.

For example, it is trivial to write a for loop that will rename files to fit a certain pattern. Since the shell itself is evaluating the loop, the LIST can be a regular expressions just like those given to any other command.

This script adds the prefix "text-" to all files in the current directory whose names end with ".txt", and moves them to the "unix/" subdirectory:

      #!/bin/bash

      for file in *.txt; do
        mv $file text/unix-$file
      done

But suppose we want to make sure no files get overwritten. This can be done with an if statement that tests for the existence of a file. The next script will do the same as the above, but also preserves any files that would have been lost by moving them to the "backup/" subdirectory:

      #!/bin/bash

      for file in *.txt; do
        if [ -e text/unix-$file ]; then
   mv text/unix-$file backup/unix-$file
 fi

        mv $file text/unix-$file
      done

Script 10: Control Structure 1
GitHub Link

Script 11: Control Structure 2
GitHub Link

Script 12: Control Structure 3

GitHub Link

Script 13: Control Structure 4
GitHub Link

Script 14: Control Structure 5
GitHub Link

Positional Parameters:

Special built-in environmental variables are positional parameters which hold command-line arguments to positions with the names 1,2,3,4, etc. which are indicated by $1,$2,$3,$4, etc. Argument $0 is the name of the script.

Script 15: Positional Parameters
GitHub Link

Select Menus:

The select construct allows easy menu generation. The syntax is quite similar to that of the for loop:

LIST is expanded, generating a list of items. The expansion is printed to standard error; each item is preceded by a number. If in LIST is not present, the positional parameters are printed, as if in $@ would have been specified. LIST is only printed once.

Upon printing all the items, the PS3 prompt is printed and one line from standard input is read. If this line consists of a number corresponding to one of the items, the value of WORD is set to the name of that item. If the line is empty, the items and the PS3 prompt are displayed again. If an EOF (End Of File) character is read, the loop exits. Since most users don't have a clue which key combination is used for the EOF sequence, it is more user-friendly to have a break command as one of the items. Any other value of the read line will set WORD to be a null string.

The read line is saved in the REPLY variable.

select var in "CHOICE1 CHOICE2"

command

done

Script 16: Select Menus 1
GitHub Link

Script 17: Select Menus 2

GitHub Link

Move many files:

Script 18: Move many files 1

GitHub Link

Script 19: Move many files 2
GitHub Link

Script 20: Monitor Service

GitHub Link

Script 21: Script for FTP Synchronization

GitHub Link

Create "ftpsynchronize.lftp" with the following text.
open -u <user>,<passwd> <remote hostname/IP>
cd temp2
lcd /root/temp2
mirror -Rn

Script 22: Backup Files
GitHub Link

Subscribe to: Posts ( Atom )

Linux Library

Pages

Saturday 31 August 2013

ELA_03_Unix_File_System

Linux Directory Structure (File System Structure) Explained with Examples

1. / – Root

2. /bin – User Binaries

3. /sbin – System Binaries

4. /etc – Configuration Files

5. /dev – Device Files

6. /proc – Process Information

7. /var – Variable Files

8. /tmp – Temporary Files

9. /usr – User Programs

10. /home – Home Directories

11. /boot – Boot Loader Files

12. /lib – System Libraries

13. /opt – Optional add-on Applications

14. /mnt – Mount Directory

15. /media – Removable Media Devices

16. /srv – Service Data

Enterprise Linux Administration (ELA_00)

ELA_01_History_of_Unix_&_Linux

History of Unix and Linux

Saturday 24 August 2013

Bash Scripting