100 VI Commands for programmers

Vi is one of the programmer’s best friend in UNIX operating systems. Never mind you’re new to Vi or not, here’s a big list of 100 useful commands, organized by topic, which will make your coder life better.

Basics

:e filename	Open filename for edition
:w	Save file
:q	Exit Vim
:w!	Write file and quit

Search

/word	Search word from top to bottom
?word	Search word from bottom to top
/jo[ha]n	Search john or joan
/\< the	Search the, theatre or then
/the\>	Search the or breathe
/\< the\>	Search the
/\< ¦.\>	Search all words of 4 letters
/\/	Search fred but not alfred or frederick
/fred\|joe	Search fred or joe
/\<\d\d\d\d\>	Search exactly 4 digits
/^\n\{3}	Find 3 empty lines
:bufdo /searchstr/	Search in all open files

Replace

:%s/old/new/g	Replace all occurences of old by new in file
:%s/old/new/gw	Replace all occurences with confirmation
:2,35s/old/new/g	Replace all occurences between lines 2 and 35
:5,$s/old/new/g	Replace all occurences from line 5 to EOF
:%s/^/hello/g	Replace the begining of each line by hello
:%s/$/Harry/g	Replace the end of each line by Harry
:%s/onward/forward/gi	Replace onward by forward, case unsensitive
:%s/ *$//g	Delete all white spaces
:g/string/d	Delete all lines containing string
:v/string/d	Delete all lines containing which didn’t contain string
:s/Bill/Steve/	Replace the first occurence of Bill by Steve in current line
:s/Bill/Steve/g	Replace Bill by Steve in current line
:%s/Bill/Steve/g	Replace Bill by Steve in all the file
:%s/\r//g	Delete DOS carriage returns (^M)
:%s/\r/\r/g	Transform DOS carriage returns in returns
:%s#<[^>]\+>##g	Delete HTML tags but keeps text
:%s/^\(.*\)\n\1$/\1/	Delete lines which appears twice
Ctrl+a	Increment number under the cursor
Ctrl+x	Decrement number under cursor
ggVGg?	Change text to Rot13

Case

Vu	Lowercase line
VU	Uppercase line
g~~	Invert case
vEU	Switch word to uppercase
vE~	Modify word case
ggguG	Set all text to lowercase
:set ignorecase	Ignore case in searches
:set smartcase	Ignore case in searches excepted if an uppercase letter is used
:%s/\<./\u&/g	Sets first letter of each word to uppercase
:%s/\<./\l&/g	Sets first letter of each word to lowercase
:%s/.*/\u&	Sets first letter of each line to uppercase
:%s/.*/\l&	Sets first letter of each line to lowercase

Read/Write files

:1,10 w outfile	Saves lines 1 to 10 in outfile
:1,10 w >> outfile	Appends lines 1 to 10 to outfile
:r infile	Insert the content of infile
:23r infile	Insert the content of infile under line 23

File explorer

:e .	Open integrated file explorer
:Sex	Split window and open integrated file explorer
:browse e	Graphical file explorer
:ls	List buffers
:cd ..	Move to parent directory
:args	List files
:args *.php	Open file list
:grep expression *.php	Returns a list of .php files contening expression
gf	Open file name under cursor

Interact with Unix

:!pwd	Execute the pwd unix command, then returns to Vi
!!pwd	Execute the pwd unix command and insert output in file
:sh	Temporary returns to Unix
$exit	Retourns to Vi

Alignment

:%!fmt	Align all lines
!}fmt	Align all lines at the current position
5!!fmt	Align the next 5 lines

Tabs

:tabnew	Creates a new tab
gt	Show next tab
:tabfirst	Show first tab
:tablast	Show last tab
:tabm n(position)	Rearrange tabs
:tabdo %s/foo/bar/g	Execute a command in all tabs
:tab ball	Puts all open files in tabs

Window spliting

:e filename	Edit filename in current window
:split filename	Split the window and open filename
ctrl-w up arrow	Puts cursor in top window
ctrl-w ctrl-w	Puts cursor in next window
ctrl-w_	Maximise current window
ctrl-w=	Gives the same size to all windows
10 ctrl-w+	Add 10 lines to current window
:vsplit file	Split window vertically
:sview file	Same as :split in readonly mode
:hide	Close current window
:­nly	Close all windows, excepted current
:b 2	Open #2 in this window

Auto-completion

Ctrl+n Ctrl+p (in insert mode)	Complete word
Ctrl+x Ctrl+l	Complete line
:set dictionary=dict	Define dict as a dictionnary
Ctrl+x Ctrl+k	Complete with dictionnary

Marks

mk	Marks current position as k
˜k	Moves cursor to mark k
d™k	Delete all until mark k

Abbreviations

:ab mail mail@provider.org

Define mail as abbreviation of mail@provider.org

Text indent

:set autoindent	Turn on auto-indent
:set smartindent	Turn on intelligent auto-indent
:set shiftwidth=4	Defines 4 spaces as indent size
ctrl-t, ctrl-d	Indent/un-indent in insert mode
>>	Indent
<<	Un-indent

Syntax highlighting

:syntax on	Turn on syntax highlighting
:syntax off	Turn off syntax highlighting
:set syntax=perl	Force syntax highlighting

Read More

TSM Quick Reference for the Backup-Archive Unix Clients

UNIX Backup-Archive Clients

Starting a TSM GUI Session

You can start a TSM GUI session, do the following:

   • Start the X Window System

   • Enter dsm (runs TSM in foreground) or dsm & (runs TSM in background)

Performing a GUI Backup

To run a GUI backup, perform the following steps:

   1. Click on Backup files and directories from the TSM window.

   2. Expand the directory tree. Select the object that you want to back up. To search or filter files, click   Search icon from the tool bar.

   3. Select the type of backup from the pull-down menu:

        o To perform an incremental backup: click on Incremental (complete)Incremental (complete)

        o To perform an incremental by date: click on Incremental (date only)

        o To perform a selective backup: click on Always backup

        o To perform an image backup: click on Image Backup

    4. To modify specific backup options click the Backup Options button located to the right of the Estimate button. The options you select are effective during the current TSM session only.

    5. Click on Backup. The Task List window displays backup processing status.

Restoring Backup Versions

To run a GUI restore, perform the following steps:

   1. Click on Restore files and directories to your system from the TSM window.

   2. Expand the directory tree. Select the object that you want to restore. To search or filter files, click the Search icon from the tool bar.

         To restore a backup set:

             a. Expand the Backup Sets directory tree object.

              b. To restore backup sets locally, expand the Local object and specify the local device or
                  filename  containing the backup set.

             c. To restore a backup set from the server, expand the Server object and select the backup set 
                  name.

   3. To modify specific restore options click the Restore Options button located to the right of the Estimate
       button. The options you select are effective during the current TSM session only.

   4. Click on Restore. Enter the information in the Restore Destination window.

    5. Click on Restore. The Task List window displays restore processing status.

Performing a GUI Archive

To archive directory or drive files you can use search and filter criteria to locate the files that match that criteria. To search or filter files during an archive operation, perform the following steps:

   1. Click on Archive files and directories into long term storage from the TSM window. The Archive 
       window displays.
   2. Expand the directory tree. Select the object that you want to archive. To search or filter files, click the 
       Search icon from the tool bar.

   3. In the Description field:

       o Enter a description, or

       o Accept the default description, or

       o Select an existing description from the drop-down list.

  4. To modify specific archive options click the Archive Options button located to the right of the Estimate   button. The options you select are effective during the current TSM session only.

  5. Click on Archive. The Task List window displays the archive processing status.

Retrieving Archives

Retrieve a file when you want to return an archive copy from the TSM server to your workstation. To retrieve an archived file:

1. Click on Retrieve from the TSM window. The Retrieve window displays.

2. Expand the directory tree. Select the directory or files you want to retrieve. To search or filter files, click the Search icon from the tool bar.

3. To modify specific retrieve options, click the Retrieve Options button located to the right of the Estimate button. The options you select are effective during the current TSM session only.

4. Click the Retrieve button. The Retrieve Destination window displays. Enter the appropriate information.

5. Click on Retrieve. The Task List window displays the retrieve processing status.

Displaying GUI Online Help

To display GUI online help, do one of the following:

• Place cursor on option or field of interest and press F1

• Select Help menu

• Click on Help button in many windows

Ending a GUI Session

To end a GUI session, do one of the following:

• Select File menu; Exit item

• Double-click on the system menu icon

• Select the system menu; close item

Starting the TSM Scheduler

To start the TSM Scheduler, do one of the following:

• Enter the dsmc schedule command, or

• Include the dsmc schedule command in your /etc/inittab file

To display information about scheduled work for your client node, enter:

dsmc query schedule

To display information about completed work for your client node: Check the dsmsched.log file in the current directory.

To stop the scheduler:

To stop the scheduler:

• Use Ctrl+C (foreground)

• Enter the kill command (background)

• Enter q.

________________________________________

Starting a TSM Command Line Session

To start a command line session, do the following:

• On the command line, change directory to the TSM directory:

o Batch mode--for a single client command, enter:

> dsmc command

o Interactive mode--for more than one client command, enter:

> dsmc or dsmc loop

tsm> command

:

tsm> quit

Performing a Command Line Backup

• To run an incremental backup of the /home file system:

• dsmc incremental /home

• To run an incremental backup by date of the /home/jones directory, enter:

• dsmc incremental -incrbydate /home/jones

• To run a selective backup of the proja file in the /home/jones directory, enter:

• dsmc selective /home/jones/proja

• To run an image backup of the /home/jones directory, enter:

• dsmc backup image /home/jones

Restoring Backup Versions

To run a command line restore:

• To restore the /home/jones/proja file to its original directory, enter:

• dsmc restore /home/jones/proja

• To restore a backup set from the /dev/rmt0 tape device, enter:

• dsmc restore backupset /dev/rmt0 -location=tape

Note: You can use wildcard characters to restore groups of files, or entire directories.

Performing a Command Line Archive

Use the archive command to archive a single file, a group of files, or all files in a directory and its subdirectories. If you want to archive all the files in a directory and its subdirectories, use the subdir=yes option. Use the deletefiles option if you want TSM to delete the files from your local workstation after they are archived and committed to storage.

For example, to archive the files in the /home/jones/proj directory and its subdirectories, enter:

dsmc archive /home/jones/proj/ -subdir=yes -deletefiles

You can use more than one file specification with the archive command. For example, to archive the /home/jones/h1.doc and /home/jones/test.doc files, enter:

dsmc archive /home/jones/h1.doc /home/jones/test.doc

Use the description option to assign a description to the archive, as follows:

dsmc archive /home/jones/h1.doc -description="Chapter 1, first version"

Retrieving Archives

You can use the retrieve command to retrieve files or directories. Enter the file name you want to retrieve and its destination. If you do not indicate a destination, the files are retrieved to their original location. For example, to retrieve the /home/jones/h1.doc file to its original directory; enter:

dsmc retrieve /home/jones/h1.doc

To retrieve this file under a new name, enter:

dsmc retrieve /home/jones/h1.doc /home/jones/h2.doc

Displaying Command Line Help

To display command line help, enter the following command:

dsmc help

________________________________________

Using the Web Client

You can use the Web client to perform backup, archive, restore, and retrieve operations from any machine, or platform that supports a Java 1.1.6-capable browser, such as Netscape Navigator 4.06 or higher, and Microsoft Internet Explorer 4.01 or higher. Using the Web client, you can back up and restore your own data, or a TSM administrator can centralize the backup or restore operations of many TSM clients. To use it, simply point your Web browser at any TSM client machine that is running the Web client. The Web client is only supported on a server that is Version 3 or higher.

All backup and restore procedures in this chapter are also supported on the Web client. The following functions are not supported by the Web client:

• Estimate

• View Policy Information

• Access Another User

• Searching and Filtering

• User Access List

• Setup Wizard

• Querying and Restoring Backup Sets

To use the Web client, perform the following steps

1. If you do not already have password files generated for TSM clients, specify passwordaccess generate in the dsm.sys options file.

2. To create the TSM password file, start the native backup-archive client by entering:

3. dsmc query session

and, when prompted, enter your user ID and password.

4. Quit from the native TSM backup-archive client.

5. Start the TSM Client Acceptor daemon by entering the dsmcad command. The TSM Web client agent service automatically starts when needed.

The only options you can use with the dsmcad command are optfile, httpport, and httpsport.

All the Web client messages are written to the Web client log file, dsmwebcl.log. Error messages are also written to the TSM error log file. The dsmwebcl.log file and the TSM error log file are located in the same directory specified by the errorlogname option.

6. To use a Web client with an enterprise management server, ensure that you were assigned an administrative user id with system privilege, policy privilege, client access, or clientowner authority. If your server does not support enterprise management, then use your node name and password when prompted by the Web client. To determine whether your server supports enterprise management, contact your server administrator.

Note: Use the revokeremoteaccess option to restrict an administrator with client access privilege from accessing your workstation through the Web client. Administrators with clientowner privilege, system privilege, or policy privilege can access the Web client.

7. You can access the TSM Web client using any browser that is Java1.1.6-capable by entering the following URL:

8. http://your_machine_name:1581

where your_machine_name is the hostname of the machine running the TSM Web client.

Port 1581 is the default port number. The administrator can set a different port number using the httpport option.

The Web client workstation can also be accessed through the TSM system administrative GUI.

If you want to use the Web client with secure socket layer (SSL) communications, which is only supported on the AIX clients, see the Appendix in Tivoli Storage Manager Installing the Clients, order number SH26-4119, for more information.

________________________________________

Summary of Commands

dsmc archive [options] filespec|"filespec"
dsmc backup image [options] filespec|"filespec"
dsmc cancel restore [options]
dsmc delete access [options]
dsmc delete archive [options] filespec|"filespec"
dsmc delete filespace [options]
dsmc help
dsmc incremental [options] [filespec|"filespec"]
dsmc loop [options]
dsmc macro macroname
dsmc query access [options]
dsmc query archive [options] filespec|"filespec"
dsmc query backup [options] filespec|"filespec"
dsmc query backupset [options] [backupsetname|filename]
dsmc query filespace [options]
dsmc query image [options] logicalvolumename|filespacename
dsmc query mgmtclass [options]
dsmc query restore [options]
dsmc query schedule [options]
dsmc query session [options]
dsmc restart restore [options]
dsmc restore [file] [options] sourcefilespec|"sourcefilespec" [destinationfilespec]
dsmc restore backupset [options] [backupsetname|filename sourcefilespec|
"sourcefilespec" [destinationfilespec]
dsmc restore image [options]sourcefilespec|"sourcefilespec" [destinationfilespec]
dsmc retrieve [options] sourcefilespec|"sourcefilespec" [destinationfilespec]
dsmc schedule [options]
dsmc selective [options] filespec|"filespec"
dsmc set access [archive|backup] filespec|"filespec" node [user] [options]
dsmc set password [oldpw newpw] [options]

Read More

TSM Client Backup Guide

TSM Backup Principles

TSM Capabilities:

The following passages are from the TSM server documentation and serve as an overview of only a small subset of capabilities that TSM offers. These are the most frequently used capabilities, but even still, only the “Backup and restore” option is the one which our TSM backup service utilizes. Our TSM backup service uses the progressive backup method (incremental forever).

Backup and restore

The backup process copies data from client workstations to server storage to ensure against loss of data that is regularly changed. The server retains versions of a file according to policy, and replaces older versions of the file with newer versions. Policy includes the number of versions and the retention time for versions. A client can restore the most recent version of a file, or can restore earlier versions.

Archive and retrieve

The archive process copies data from client workstations to server storage for long-term storage. The process can optionally delete the archived files from the client workstations. The server retains archive copies according to the policy for archive retention time. A client can retrieve an archived copy of a file.

File-level Backup Policies:

Policies available to you are defined with four parameters. The table below outlines the use of each parameter:

TSM marks versions of a file as inactive once a new “active” version is backed up, once the file is deleted, or when it is excluded. Once a file is deleted or excluded, TSM marks all versions inactive. If the file exists on the host and it is not excluded from backups, there will always be an active version on TSM.


Our policy information can be found in the “Data Retention Form”. Please keep in mind that these policies are for data protection only. Any need for archiving, long-term retention, or bare system restores will not be addressed by these policies.

When registering for the service, you must specify one default retention plan. You may configure your client to back up certain files or filesystems under non-default policies by creating an include-exclude list. Refer to the include-exclude list documentation.

TSM Client Installation and Configuration

Client components which are normally installed are:


• Client Native GUI

• Client Web GUI

• Client Command Line Interface

• Client Acceptor Daemon and Scheduler


Refer to client readme as some of these features are not included with some platforms (i.e. Netware does not have a native GUI).

For quick and easy installation procedures, please see the  below e:

http://publib.boulder.ibm.com/infocenter/tivihelp/v1r1/topic/com.ibm.itsmc.doc/baclients.html

Though installation and configuration can be pretty straightforward, there are many other options available in TSM to fine-tune your backups for performance and space-saving benefits.  Most clients should operate well with defaults, but if you run into bottlenecks, or your clients are eratic, consider setting some advanced configuration options.

Advanced Client Configuration

Unix/Linux/Mac OS clients rely on the dsm.opt and dsm.sys files for configuration options.
Windows/Netware clients rely solely on dsm.opt files for configuration options.
For official documentation for all available options, refer to:

http://publib.boulder.ibm.com/infocenter/tivihelp/v1r1/topic/com.ibm.itsmc.doc/baclients.html

Include/Exclude lists dictate what to backup, and what policies should files be placed under.


Exclude Statement Structures

exclude [file] Excludes files from backups and archives

exclude.archive [file] Excludes file from archives

exclude.fs [filesystem] Excludes filesystem from backups and archives

exclude.dir [directory] Excludes directory from backups and archives

exclude.compression [file] Excludes file from compression


Include Statement Structures

include [file] [class]
 Includes file for backups and archives. If class is specified, then it binds the file to that management class. Otherwise, it uses the default.

include.archive [file] [class]
Includes file for archives only. If class is specified, then it binds the file to that management class. Otherwise, it uses the default.

include.compression [file]
Includes file for compression only if “Compression yes” is specified in options file.

A note regarding the use of wildcards for include/exclude processing (Unix syntax):

/… : matches 0 or more directories

* : matches and characters/files

? : matches any one character

[ ] : matches a character class enumeration or range

If you exclude a file with /… then all subdirectory information is backed-up. The files contained in those directories are not. For spaces in file names, use single-quotation marks.


A note regarding the order of exclude/include lists (Unix syntax):

Tivoli Storage Manager evaluates all exclude.fs and exclude.dir statements first (regardless of their position within the include-exclude list), and removes the excluded file spaces, directories, and files from the list of objects available for processing. All other include-exclude statements are processed from the bottom of the list up. Therefore, it is important to enter all your include-exclude statements in the proper order. For example, in the following include-exclude list the includefile.txt file is not backed up:


include /home/usr/includefile.txt

exclude /home/usr/.../*

However, in the following include-exclude list the includefile.txt file is backed up:

exclude /home/usr/.../*

include /home/usr/includefile.txt

For more detailed information on the usage of include-exclude lists, refer to the official documentation:

http://publib.boulder.ibm.com/infocenter/tivihelp/v1r1/topic/com.ibm.itsmc.doc/baclients.html:

TSM Client Operations

This section details the client operations that you may encounter after the installation. Maintenance, backups, and restores are covered.


Regular Client Maintenance:

Regular maintenance tasks include:

• Checking the backup log for files that have consistently failed backing up. Files with backup errors are good candidates for exclusion or for corrective measures. Scheduled backup logs and error logs are available on all clients:

Windows: C:\Program Files\Tivoli\TSM\baclient\

Netware: SYS:\tivoli\tsm\client\ba\

Linux: /opt/tivoli/tsm/client/ba/bin/

Solaris: /opt/tivoli/tsm/client/ba/bin/

AIX: /usr/opt/tivoli/tsm/client/ba/bin/

Mac OS X: /Library/Logs/tivoli/tsm/

TSM Scheduled vs. Manual Backups:



Scheduled backups are handled through the ManagedServices client option along with server-defined schedules. The legacy way of handling schedules was through the “dsmc sched” command. The dsmcad does the following to pick up its scheduled backup time:

1. Once the administrator runs dsmcad, it sits in the background.

2. dsmcad kicks off the scheduler after a minute, and the scheduler queries the server for its next scheduled action (either another query or incremental backup). It logs to the dsmsched.log and exits.

3. dsmcad remains idle until its next scheduled activity.


Manual backups are handled either through the GUI or commandline.

Command line sample:

# dsmc incremental “/usr/*”

The incremental backup performs versioning-based backups. A selective backup forces all files to be backed up:
# dsmc selective “/usr/*”

TSM Client Restores - Techniques and Best Practices:

TSM classifies restores into two categories: no-query restore, and traditional restores.

Traditional Restores

Traditional restores involve the specification of a particular file or files to be restored. The GUI offers the easiest method to identify precise versions of files that you need restored. However, the command line also offers traditional restores.

Example:

#dsmc restore “/home/*/foo.*” –pick -inactive

No-Query Restores

A no-query restore (also known as a restartable restore) offers disaster recovery for directories and filesystems. A single wild-card specification identifies the restore as a no-query restore.

Example:

#dsmc restore “/var/tsm/*” –subdir=yes

Best Practices

Below are recommendations for improved restore times:

• For large restores, familiarize yourself with the command line client (dsmc). It is a more powerful mechanism for file restoration than the GUI, though it is less user friendly.

• For point-in-time restores, the GUI offers the easiest method to identify which files you need precisely.

• For large restores, try running multiple restore sessions at the same time. Though restores may compete for tape resources, each session will run to completion so less intervention is necessary.

Client Commands

Query backup and arch:

>/usr/bin/sudo /usr/bin/dsmc q arch –subdir=1 –fromnode=

>/usr/bin/sudo /usr/bin/dsmc q arch –subdir=1 –fromnode=

-fromdate=08/20/2006 -fromtime=11:00 -todate=08/20/2006 -totime=17:00 /db/abc/oradata01/

>/usr/bin/sudo /usr/bin/dsmc q back –subdir=1 –fromnode= /db/abc/oradata01/

>/usr/bin/sudo /usr/bin/dsmc q back –ina –subdir=1 –fromnode= /db/abc/oradata01/

Backup and arch:

>/usr/bin/sudo /usr/bin/dsmc arch –subdir=1 –desc=” OAPR_hotdb” /db/abc/oradata01/ -archmc=”OAPRHOT”

>/usr/bin/sudo /usr/bin/dsmc arch –subdir=1 "” /db/abc/oradata01/ -archmc=”OAPRHOT” -desc=”OAPR_hotdb_"%%$DATE._%%TIME

>/usr/bin/sudo /usr/bin/dsmc incremental –subdir=1 /db/abc/oradata01/

Restore and retrieve:

> /usr/bin/sudo /usr/bin/dsmc retrieve -subdir=yes -fromnode=- desc="ABCD_COLD_20060820_11:00" /db/ABCD/oradata02/ /db/ABCD/oradata02/

>/usr/bin/sudo /usr/bin/dsmc retrieve -subdir=yes -fromnode=-fromdate=08/20/2006 -fromtime=11:00 -todate=08/20/2006 -totime=17:00 /db/ABCD/oradata02/ /db/ABCD/oradata02/

> /usr/bin/sudo /usr/bin/dsmc restore -subdir=yes -fromnode=- desc="ABCD_COLD_20060820_11:00" /db/ABCD/oradata02/ /db/ABCD/oradata02/

To set Access to a node

>set access arch * node

or

>set access arch * *

To delete Access

type

>q access (to see the list then type)

>delete access

then choose the list to delete. such 1,2,3.

To see what is backedup on a client.

>q files

>q backup d:\

>q node

>q mgmt (policydomain) active * f=d

Read More

Prominent UNIX tools

Much like a vernacular, the universe of UNIX tools changes almost perpetually. New tools crop up frequently, while others are eternally modernized and adapted to suit emerging best practices. Certain tools are used commonly; others are used more infrequently. Some tools are perennial; occasionally, some are obsoleted outright. To speak UNIX fluently, you have to keep up with the "lingo."

 Prominent UNIX tools

Name	Purpose
Cygwin	A UNIX-like shell and build environment for the Windows® operating system.
fish	A highly interactive shell with automatic expansion and colored syntax for command names, options, and file names.
locate	Build and search a database of all files
rename	Rename large collections of files en masse
rsync	Efficiently synchronize files and directories, locally and remotely
Screen	Create and manage virtual, persistent consoles
Squirrel	A cross-platform scripting shell
tac	Print input in reverse order, last line first (tac is the reverse of cat)
type	Reveal whether a command is an alias, an executable, a shell built in, or a script
wget	Download files using the command line
zsh	An advanced shell featuring automatic completion, advanced redirection operands, and advanced substitutions

Read More