Monday 23 December 2013

PS_06_Perl - Associative Arrays (Hashes)

Associative Arrays (Hashes)

A hash (or associative array) is an unordered set of key/value pairs whose elements are indexed by their keys. Hash variable names have the form %foo.

Hash Variables and Literals
A literal representation of a hash is a list with an even number of elements (key/value pairs, remember?).
%foo = qw( fred wilma barney betty );
%foo = @foolist;

To add individual elements to a hash, all you have to do is set them individually:
$foo{fred} = “wilma”;
$foo{barney} = “betty”;

You can also access slices of hashes in a manner similar to the list case:
@foo{“fred”,”barney”} = qw( wilma betty );

Hash Functions
The keys function returns a list of all the current keys for the hash in question.
@hashkeys = keys(%hash);

As with all other built-in functions, the parentheses are optional:
@hashkeys = keys %hash;

This is often used to iterate over all elements of a hash:
foreach $key (keys %hash) {
print $hash{$key}.”\n”;
}

In a scalar context, the keys function gives the number of elements in the hash. Conversely, the values function returns a list of all current values of the argument hash:
@hashvals = values(%hash);

The each function provides another means of iterating over the elements in a hash:
while (($key, $value) = each (%hash)) {
statements;
}

You can remove elements from a hash using the delete function:
delete $hash{‘key’};

PS_05_Perl - Indexed Arrays (Lists)

Indexed Arrays (Lists)

A list is an ordered set of scalar data. List names follow the same basic rules as for scalars. A reference to a list has the form @foo.

List literals
List literals consist of comma-separated values enclosed in parentheses:
(1,2,3)
(“foo”,4.5)

A range can be represented using a list constructor function (such as “..”):
(1..9) = (1,2,3,4,5,6,7,8,9)
($a..$b) = ($a, $a+1, … , $b-1,$b)

In the case of string values, it can be convenient to use the “quote-word” syntax
@a = (“fred”,”barney”,”betty”,”wilma”);
@a = qw( fred barney betty wilma );

Accessing List Elements
List elements are subscripted by sequential integers, beginning with 0
$foo[5] is the sixth element of @foo

The special variable $#foo provides the index value of the last element of @foo. A subset of elements from a list is called a slice.
@foo[0,1] is the same as ($foo[0],$foo[1])

You can also access slices of list literals:
@foo = (qw( fred barney betty wilma ))[2,3]

List operators and functions
Many list-processing functions operate on the paradigm in which the list is a stack. The highest subscript end of the list is the “top,” and the lowest is the bottom.
push Appends a value to the end of the list
push(@mylist,$newvalue)
pop Removes the last element from the list (and returns it)
pop(@mylist)
shift Removes the first element from the list (and returns it)
shift(@mylist)
unshift Prepends a value to the beginning of the list
unshift(@mylist,$newvalue)
splice Inserts elements into a list at an arbitrary position
splice(@mylist,$offset,$replace,@newlist)

The reverse function reverses the order of the elements of a list
@b = reverse(@a);

The sort function sorts the elements of its argument as strings in ASCII order. You can also customize the sorting algorithm if you want to do something special.
@x = sort(@y);

The chomp function works on lists as well as scalars. When invoked on a list, it removes newlines (record separators) from each element of its argument.

Sunday 22 December 2013

PS_04_Perl - Control_Structures

Control Structures

Statement Blocks
A statement block is simply a sequence of statements enclose in curly braces:
{
first_statement;
second_statement;
last_statement
}

Conditional Structures (If/elsif/else)
The basic construction to execute blocks of statements is the if statement. The if statement permits execution of the associated statement block if the test expression evaluates as true. It is important to note that unlike many compiled languages, it is necessary to enclose the statement block in curly braces, even if only one statement is to be executed.

The general form of an if/then/else type of control statement is as follows:
if (expression_one) {
true_one_statement;
} elsif (expression_two) {
true_two_statement;
} else {
all_false_statement;
}

For convenience, Perl also offers a construct to test if an expression is false:
unless (expression) {
false_statement;
} else {
true_statement;
}

Note that the order of the conditional can be inverted as well:
statement if (expression);
statement unless (expression);

The “ternary” operator is another nifty one to keep in your bag of tricks:
$var = (expression) ? true_value : false_value;

It is equivalent to:
if (expression) {
$var = true_value;
} else {
$var = false_value;
}


Loops
Perl provides several different means of repetitively executing blocks of statements.

While
The basic while loop tests an expression before executing a statement block
while (expression) {
statements;
}

Until
The until loop tests an expression at the end of a statement block; statements will be executed until the expression evaluates as true.
until (expression) {
statements;
}

Do while
A statement block is executed at least once, and then repeatedly until the test expression is false.
do {
statements;
} while (expression);

Do until
A statement block is executed at least once, and then repeatedly until the test expression is true.
do {
statements;
} until (expression);

For
The for loop has three semicolon-separated expressions within its arentheses. These expressions function respectively for the initialization, the condition, and re-initialization expressions of the loop. The for loop
for (initial_exp; test_exp; reinit_exp) {
statements;
}

This structure is typically used to iterate over a range of values. The loop runs until the
test_exp is false.
for ($i; $i<10;$i++) {
print $i;
}

Foreach
The foreach statement is much like the for statement except it loops over the elements of a list:
foreach $i (@some_list) {
statements;
}
If the scalar loop variable is omitted, $_ is used.

Labels
Any statement block can be given a label. Labels are identifiers that follow variable naming rules. They are placed immediately before a statement block and end with a colon:
SOMELABEL: {
statements;
}
You can short-circuit loop execution with the directives next and last:
· next skips the remaining statements in the loop and proceeds to the next iteration (if any)
· last immediately exits the loop in question
· redo jumps to the beginning of the block (restarting current iteration)
Next and last can be used in conjunction with a label to specify a loop by name. If the label is omitted, the presumption is that next/last refers to the innermost enclosing loop. Usually deprecated in most languages, the goto expression is nevertheless supported by Perl. It is usually used in connection with a label
goto LABEL;
to jump to a particular point of execution.

Wednesday 11 December 2013

Setting up a PXE boot server

Setting up a PXE boot server on a Linux Machine

0) The first thing to note is that you need to setup your own mini-network that is completely disconnected from the network, since part of this process requires setting up a DHCP server which could conflict with the corporate DHCP server if they were both running on the same network simultaneously. So get yourself a switch from IT up front. You do *NOT* need the switch immediately, so just put it aside until I mention it again
later on.

1) The next step is to choose a box to be the PXE boot server. This can really be any box at all, as long as you have a NIC in it that works reliably under Linux. For the purposes of this documentation, I'm going to assume that you've loaded Fedora Core 4 on this box (do that now, if you've not already). Get this box onto the network with DHCP (just like a normal installation).

2) Next you'll need to install the following packages (which ship with FC4 already, so if you did an 'everything' OS install, you should have them already. If not, you can install them easily with yum):
tftp-server
dhcp
httpd
syslinux

If you use yum to install them, then it will be generally alot easier:
yum install tftp-server dhcp httpd syslinux
answer Y to all dependency/installation questions.

3) Now you need to setup the DHCP server. With the FC4 RPM for dhcp, all you need to do is create /etc/dhcpd.conf with the following contents:

ddns-update-style interim;
subnet 192.168.0.0 netmask 255.255.255.0 {
range 192.168.0.10 192.168.0.254;
default-lease-time 3600;
max-lease-time 4800;
option routers 192.168.0.1;
option domain-name-servers 192.168.0.1;
option subnet-mask 255.255.255.0;
option domain-name "llama.net";
option time-offset -8;
}

host llama0 {
hardware ethernet 04:4B:80:80:80:03;
fixed-address 192.168.0.254;
option host-name "llama0";
filename "pxelinux.0";
}

In a nutshell, this sets up a DNS server that will assign IP address 192.168.0.254 to your client box that has MAC address 04:4B:80:80:80:03 assigned to its PXE-boot capable NIC. Another thing to note is that we're reserving the private 192.168 subnet for this setup. The only thing you need to change in the above, is the MAC address to match that of the NIC on your client box.

4) Next you need to activate tftp within xinetd. All that is neccesary is to change disable=yes to disable=no in /etc/xinetd.d/tftp . Then restart xinetd. For future reference, the tftp RPM for FC4 stores its servable content under /tftpboot.

5) Now we need to setup your PXE server to use a static IP on the new private subnet. Create the file /etc/sysconfig/network-scripts/ifcfg-eth0.static with the following contents:

DEVICE=eth0
BOOTPROTO=STATIC
ONBOOT=no
TYPE=Ethernet
IPADDR=192.168.0.2
NETMASK=255.255.255.0
GATEWAY=192.168.0.1

6) Now we need to setup the PXE boot environment on the server. To do this, you need to have either the Linux distribution that you wish to install over PXE either in CD format, or all the content of the CDs available on the network.
On the first CD of every RH/FC distribution there is a subdirectory called 'isolinux'. In that directory you will find two files, vmlinuz and initrd.img. These are the kernel & initrd.img that the RH/FC bootable CDs use to get the installer (anaconda) booted for performing the installation. Copy both of those files into /tftpboot and make sure that they are world readable. If you are planning to allow more than one version/distribution to be PXE boot installable, then you should rename both files so that its clear that they are for whatever version/distribution they came from (such as vmlinuz-RHEL4, initrd-RHEL4).

Next, you need the actual pxe boot linux kernel (what is actually run immediately after your PXE boot client box gets a DHCP lease). In this case, that file is pxelinux.0, and is part of the syslinux RPM. For FC4, you can find it at /usr/lib/syslinux/pxelinux.0. Copy that file into /tftpboot and make sure that it is world readable.

7) Next we need to configure pxelinux. First create the directory /tftpboot/pxelinux.cfg (and make it world readable). Inside that directory you need to create a number of zero size files (use touch):
01-04-4B-80-80-80-03
C
C0
C0A
C0A8
C0A80
C0A800
C0A800F
C0A800FE
01-04-4B-80-80-80-03

The first 8 are the hex representation of the 192.168.0.254 IP address that your PXE boot client will be assigned. The permutations allow a broader IP subnet to be searched first for matches. The last entry is the MAC address of your PXE boot client's NIC (with dashes substituted for the colons), with '01' pre-pended. The "01" at the front represents a hardware type of Ethernet, so pxelinux.0 see's the configuration string as an IP address.

8) Now create the default pxelinux configuration inside the new file
/tftpboot/pxelinux.cfg/default:
prompt 1
default linux
timeout 100

label linux
kernel vmlinuz
append initrd=initrd.img ramdisk_size=9216 noapic acpi=off


9) Now you need to put the full contents of your Linux distro (all CDs) somewhere on disk. I put it under /tftpboot/RHEL4U1. In order to allow for installation over HTTP (apache), edit /etc/httpd/conf/httpd.conf and add the following:
<Directory /tftpboot/RHEL4U1>
Options Indexes
AllowOverride None
</Directory>
Alias /linux /tftpboot/RHEL4U1


10) At this stage, you're ready to hook up the switch. You should have CAT5 running between the switch & the PXE boot server, and the client box.

11) On the PXE boot server, bring down your DHCP network connected eth0 (ifdown eth0), disconnect the CAT5 connected to the network, and plug in the cat5 connected to your private switch. Now bring up the static IP for the PXE server with (ifup eth0.static). You can verify that it came up successfully by verifying that you have IP address 192.168.0.2 in ifconfig.

12) Now start dhcpd & apache and activate tftp by running the following:
service dhcpd start
service xinetd restart
service httpd start

and verify that they are all in your process list.

13) Plug the PXE client box's CAT5 into the switch, and verify that the NIC appears first in the BIOS boot order. (re)boot and you should get a DHCP lease, and start booting successfully off the network.

14) When you get into the RH/FC installer which asks you for the install method, choose HTTP. Fill in 192.168.0.2 for the name, and 'linux' for the path, and you should be all set.

15) If you run into any problems, check /var/log/messages for errors (that's where all dhcp & tftp stuff will get logged). /var/log/httpd is where apache logs, but if you get that far, your problem is an apache configuration/setup issue, and not a PXE boot issue.


http://wizpert.com/mallikbheesetti

Thursday 28 November 2013

ELA_32_Service Management

Service Management: 

A Linux service is an application (or set of applications) that runs in the background waiting to be used, or carrying out essential tasks. I've already mentioned a couple of typical ones (Apache and MySQL). You will generally be unaware of services until you need them.

Let's start by looking at how the system is set up, and in particular at the directory /etc/rc.d. Here you will find either a set of files named rc.0, rc.1, rc.2, rc.3, rc.4, rc.5, and rc.6, or a set of directories named rc0.d, rc1.d, rc2.d, rc3.d, rc4.d, rc5.d, and rc6.d. You will also find a file named /etc/inittab. The system uses these files (and/or directories) to control the services to be started.

The boot process uses these parameters to identify the default runlevel and the files that will be used by that runlevel. In this example, runlevel 4 is the default and the scripts that define runlevel 4 can be found in /etc/rc.d/rc.4.

And what is a runlevel? You might assume that this refers to different levels that the system goes through during a boot up. Instead, think of the runlevel as the point at which the system is entered. Runlevel 1 is the most basic configuration (simple single user access using an text interface), while runlevel 5 is the most advanced (multi-user, networking, and a GUI front end). Runlevels 0 and 6 are used for halting and rebooting the system.

There are, however, differences between Linux distributions. For instance, Fedora uses runlevel 5 for X-based logins, whereas Slackware uses runlevel 4 to do the same job. Therefore, you should check your documentation before making any changes. This table shows a generic list of configurations (and some examples of different distros) taken from Linux - The Complete Reference (R.Peterson, Osbourne/McGraw-Hill).

Run Level    Generic                                                         Fedora Core       
0                 Halt                                 
                                Halt           
1                 Single-user mode               
                              Single-user mode   
2                 Basic multi-user mode (without networking)     User definable (Unused)
3                 Full (text based) multi-user mode        
            Multi-user mode
4                 Not used                   
                                      Not used
5                 Full (GUI based) multi-user mode         
            Full multi-user mode (with an X-based login screen)
6                 Reboot                       
                                     Reboot

Features:
 1. Start|Stop|Adjust runlevels of services
 2. Three tools are available
  a. 'chkconfig' - Shell
  b. 'ntsysv' - TUI
  c. 'system-config-services' - GUI

Tasks:
 1. 'chkconfig' - manages both: 'SYSV' & 'XINETD'
  a. 'cknconfig' - enumerates all services
  b. '--list vsftpd' - enumerates runlevel information for service 'vsftpd'
Note: '/etc/init.d' - services repository
  c. '--level 2345 vsftpd off'
  d. 'chkconfig vsftpd on | off' - synonym for run-levels 2-5
  e. 'chkconfig tftp on' enables XINETD-controlled service: 'tftp'
Note: XINETD-controlled services are automatically enabled | disabled by 'chkconfig'
Note: However, SYSV-controlled services are NOT automatically started | stopped
Note: Use 'service service_name start|stop' to control service

 2. 'ntsysv' - defaults to managing services in the current run-level
Manages both 'SYSV' and 'XINETD' services
  a. 'ntsysv --level 35' - influences ONLY the levels specified on the CLI
Note: 'ntsysv' will NOT change the other, unspecified, run-levels

 3. 'system-config-services' - GUI - Manages 'SYSV' and 'XINETD' services

ELA_31_LFTP

LFTP :

Description
LFTP is a sophisticated file transfer program with command line interface. It supports FTP, HTTP, FISH, SFTP, HTTPS and FTPS protocols. GNU Readline library is used for input.

Every operation in lftp is reliable, that is any non-fatal error is handled and the operation is retried automatically. So if downloading breaks, it will be restarted from the point automatically. Even if ftp server does not support REST command, lftp will try to retrieve the file from the very beginning until the file is transferred completely. This is useful for dynamic-ip machines which change their IP addresses quite often, and for sites with very bad internet connectivity.

If you exit lftp when some jobs are not finished yet, lftp will move itself to nohup mode in background. The same happens when you have a real modem hangup or when you close an xterm.

lftp has shell-like command syntax allowing you to launch several commands in parallel in background (&). It is also possible to group commands within () and execute them in background. All background jobs are executed in the same single process. You can bring a foreground job to background with ^Z (c-z) and back with command `wait' (or `fg' which is alias to `wait'). To list running jobs, use command `jobs'. Some commands allow redirecting their output (cat, ls, ...) to file or via pipe to external command. Commands can be executed conditionally based on termination status of previous command (&&, ||).

Features:

 1. Interactive (Shell-like) & Non-interactive modes
 2. Scriptable
 3. Serrvers supported: FTP, FTPS, SSH(SFTP), HTTP, etc.
 4. Mirroring of content: forward (download) & reverse (upload)
 5. Regular expressions
 6. Job Engine

Tasks:
 1. Use 'lftp' to connect to VSFTPD
  a. 'lftp localhost' && 'open -u mallik'
Note: LFTP batches authentication commands and submits when control-channel commands such as 'ls' are received
---- Connecting to localhost (127.0.0.1) port 21 - (no connection)
<--- 220 Welcome to HINDUX FTP service. - (traffic from server to client)
---> FEAT - (traffic from client to server)

 2. Use 'lftp' to connect and mirror content
  a. 'mirror temp*' - forward mirror - downloads content from server to client
  b. 'mirror -Rv *' - reverse mirror - puts content on server from client

 3. Run external commands with '!command'
  a. '!bash' - launches an instance of BASH SHELL from within 'lftp'
  b. 'exit' - returns to 'lftp'

 4. Test rate-limiting with 'vsftpd'
  a. 'local_max_rate=1000' - B/s (Bytes per second)

 5. Job Management - Backgrounging
  a. Use: 'CTRL-Z' to background jobs
  b. Use: 'jobs' to view progress of jobs
  c. 'fg job_num' to foreground a specific job

 6. Explore LFTP environment
  a. '/etc/lftp.conf' - system-wide config file

 7. Connect using 'lftp' to: SSH & HTTP servers
  a. 'lftp http://192.168.0.100/rhel6'
  b. 'lftp -u mallik sftp://192.168.0.100'
 

ELA_30_VSFTPD (Very Secure File Transfer Protocol Daemon) Configuration

Very Secure File Transfer Protocol Daemon (VSFTPD) Configuration:

About vsftpd
vsftpd is a GPL licensed FTP server for UNIX systems, including Linux. It is secure and extremely fast. It is stable. Don't take my word for it, though. Below, we will see evidence supporting all three assertions. We will also see a list of a few important sites which are happily using vsftpd. This demonstrates vsftpd is a mature and trusted solution.
Features
Despite being small for purposes of speed and security, many more complicated FTP setups are achievable with vsftpd! By no means an exclusive list, vsftpd will handle:

    Virtual IP configurations
    Virtual users
    Standalone or inetd operation
    Powerful per-user configurability
    Bandwidth throttling
    Per-source-IP configurability
    Per-source-IP limits
    IPv6
    Ananymous (default) and user-based FTP sessions
    SSL support (provided by SSH) no need for VSFTPD
    Does not permit 'root' or 'service accounts' access by default
    Does not currently support IPv4 & IPv6 simultaneously with the same daemon
    Encryption support through SSL integration  etc...

Online source / docs
Browse vsftpd's online source tree - including documentation. In particular, note the content of the EXAMPLE subdirectory. Also, here is an HTML version of the manual page which lists all vsftpd config options.

Steps:
1. Install vsftpd
yum install vsftpd

2. Enable vsftpd service
chkconfig vsftpd on

3. Start vsftpd service
service vsftpd start

4. Check ftp port
netstat -ntl | grep 21

5. Test ftp server access
FTP server is running and anonymous access is enabled by default

6. Disable anonymous access
Edit /etc/vsftd/vsftpd.conf
anonymous_enable=NO

7. Enable dual logging
dual_log_enable=YES
It created /var/log/vsftpd.log file which uses vsftpd log format

8. Enable server time for display of files and directories
use_localtime=YES

Tasks:
 1. Install using 'yum'
 2. Enable vsftpd in multi-user runlevels
  a. 'chkconfig vsftpd on'
 3. 'service vsftpd start'
 4. Disable Anonymous access
 5. Test local user access and update SELinux configuration
  a. 'getsebool -a | grep ftp' - dumps FTP-related SELinux booleans
  b. 'setsebool -P ftp_home_dir=1'
Note: RHEL6 enables SELinux in 'enforcing' mode, requiring a slight change to the booleans to permit VSFTPD or any FTPD daemon to transition user into their: $HOME directory

 6. Enable Dual-Logging
  a. 'dual_log_enable=YES'
 7. Enable server time for display of files/directories
  a. 'use_localtime=YES'
Note: 'man vsftpd.conf' for useful directives that apply to your application

Wednesday 30 October 2013

PS_03_Perl - Basic_I/O_&_Operators

Basic I/O
The easiest means to get operator input to your program is using the “diamond” operator:
$input = <>;
The input from the diamond operator includes a newline (\n). To get rid of this pesky character, use either chop() or chomp(). chop() removes the last character of the string, while chomp() removes any line-ending characters (defined in the special variable $/). If no argument is given, these functions operate on the $_ variable. To do the converse, simply use Perl’s print function:
print $output.”\n”;

Basic Operators
Arithmetic
Example Name Result
$a + $b Addition Sum of $a and $b
$a * $b Multiplication Product of $a and $b
$a % $b Modulus Remainder of $a divided by $b
$a ** $b Exponentiation $a to the power of $b

String
Example Name Result
$a . “string” Concatenation String built from pieces
$a string” Interpolation String incorporating the value of $a
$a x $b Repeat String in which $a is repeated $b times

Assignment
The basic assignment operator is “=”: $a = $b.
Perl conforms to the C idiom that lvalue operator= expression is evaluated as: lvalue = lvalue operator expression
So that $a *= $b is equivalent to $a = $a * $b
$a += $b $a = $a + $b
This also works for the string concatenation operator: $a .= “\n”

Autoincrement and Autodecrement
The autoincrement and autodecrement operators are special cases of the assignment operators, which add or subtract 1 from the value of a variable:
++$a, $a++ Autoincrement Add 1 to $a
--$a, $a-- Autodecrement Subtract 1 from $a

Logical
Conditions for truth: Any string is true except for “” and “0” Any number is true except for 0 Any reference is true Any undefined value is false
Example Name Result
$a && $b And True if both $a and $b are true
$a || $b Or $a if $a is true; $b otherwise
!$a Not True if $a is not true
$a and $b And True if both $a and $b are true
$a or $b Or $a if $a is true; $b otherwise
not $a Not True if $a is not true
Logical operators are often used to “short circuit” expressions, as in:
open(FILE,”< input.dat”) or die “Can’t open file”;

Comparison
Comparison Numeric String Result
Equal == eq True if $a equal to $b
Not equal != ne True if $a not equal to $b
Less than < lt True if $a less than $b
Greater than > gt True if $a greater than $b
Less than or equal <= le True if $a not greater than $b
Comparison <=> cmp 0 if $a and $b equal
1 if $a greater
-1 if $b greater

Operator Precedence
Perl operators have the following precedence, listed from the highest to the lowest, where operators at the same precedence level resolve according to associativity:
Associativity Operators Description
Left Terms and
list operators
Left -> Infix dereference operator
++
--
Auto-increment
Auto-decrement
Right
Right
Right
\
! ~
+ -
Reference to an object (unary)
Unary negation, bitwise complement
Unary plus, minus
Left
Left
=~
!~
Binds scalar to a match pattern
Same, but negates the result
Left * / % x Multiplication, Division, Modulo, Repeat
Left + - . Addition, Subtraction, Concatenation
Left >> << Bitwise shift right, left
< > <= >=
lt gt le ge
Numerical relational operators
String relational operators
== != <=>
eq ne cmp
Numerical comparison operators
String comparison operators
Left & Bitwise AND
Left | ^ Bitwise OR, Exclusive OR
Left && Logical AND
Left || Logical OR
.. In scalar context, range operator
In array context, enumeration
Right ?: Conditional (if ? then : else) operator
Right = += -= etc Assignment operators
Left ,
=>
Comma operator, also list element
separator
Same, enforces left operand to be string
Right not Low precedence logical NOT
Right and Low precedence logical AND
Right or xor Low precedence logical OR
Parentheses can be used to group an expression into a term. A list consists of expressions, variables, or lists, separated by commas. An array variable or an array slice many always be used instead of a list.