Linux CMD for daily use

work

check log

  1. Display whole log file content:

    1
    $ cat target.log

  2. Search key word in log file, and return lines containing keyword:

    1
    2
    3
    $ cat target.log |grep "keyword"
    or
    $ grep -i "keyword" target.log

    these two method return same result

  3. Check the most recent log (the tail lines of log file):

    1
    2
    3
    $ cat target.log |tail -n 200   # check the last 200 lines of log file
    or
    $ tail -200f target.log   # show last 200 lines and continue printing new logs

ping

  1. To test whether the target host is accessible, mainly has two ways:
    1
    2
    $ ping <host addr>    # sample address: 192.168.1.21
    $ ping6 <host addr>   # ipv6 address
  2. ping + paras:
    1
    2
    3
    4
    5
    6
    7
    $ ping -a <addr>  # 把地址解析成主机名 (NetBios名)
                      # e.g. ping -a 192.168.1.21 returns
                      # Pinging iceblood.yofor.com [192.168.1.21] with 32 bytes of data
                      # so the NetBios name is iceblood.yofor.com with ip=192.168.1.21
    $ ping -t <addr>  # continue pinging the addr
                      # press Ctl+Break to show statistic info and continue pinging
                      # Ctl+C to stop

For more: 从ping和ping6说起

file transport (scp)

  1. copy file from path1 to path2
    1
    2
    3
    4
    5
    $ scp path1 username@targetAddr:path2
    # sample:
    $ scp /home/usrsby/xxx.jar root@10.136.40.75:/home/usrsby
    # while using jumper as proxy, file has to be transported twice
    $ scp /home/usrsby/xxx.jar node-image-name:/opt/xxx/xx-xx/xxx-server/repository/com/xxx/x/xx-service-configuration/1.0-SNAPSHOT/
  2. download file from remote vm:
    1
    2
    3
    4
    5
    # Step1: copy target file from vm to jumper in same way as above
    # Step2: download file to jumper, execute in jumper
    $ scp username@remoteAddr:/remotePath/file folder_in_jumper
    # Sample:
    $ scp root@10.136.40.75:/home/usrsby/xx.log /home/usrsby
  3. when copy a folder:
    Append -r after scp like:
    1
    2
    $ scp -r /tmp/tempA/ root@10.127.40.25:/tmp/usrsby/ # copy tempA folder to usrsby
    # !notice: do not dismiss "/" at the end of target path, not same as copying single file

For more: linux利用scp远程上传下载文件/文件夹

time

to show current local time:

1
$ date

grep, awk & sed

AWSOME specific summary! –> A brief introduction to grep, awk & sed

  1. grep (Global Regular Expression Print) is used to search for specific terms in a file.
    1
    2
    3
    4
    5
    $ grep test testscript  # grep keyword filename
    just for test
    $ grep test * # grep keyword all_files
    testCase:test
    testscript:just for test
    If multiple keywords are selected:
    1
    2
    3
    $ less filename | grep -E "kw1|kw2"
    # or
    $ grep -E "kw1|kw2" filename
    grep and save to a new file:
    1
    $ grep -E "ERROR|CRIT|WARN" xxxxx.log > /path/<new-file>.txt
  2. awk is a text pattern scanning and processing language, which is created by Aho, Weinberger & Kernighan. awk is mostly used for data extraction and reporting (dealing with .csv files).
    1
    2
    3
    4
    5
    6
    # print 1st and 4th column
    $ awk '{print $1, $4}' file.txt
    #
    # same with 'cat file.txt'
    $ awk '{print $0}' file.txt
    # for more see the reference
  3. sed refers to Stream Editor. It can perform text transformations on a given file or an input stream.
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    # replace the 1st 'test' with 'text' in each line
    sed 's/test/text/' file.txt
    #
    # replace all 'test' with 'text' in each line
    sed 's/test/text/g' file.txt
    #
    # replace the 2nd 'test' with 'text' in each line
    sed 's/test/text/2' file.txt
    #
    # replace all 'test' from the 2nd to the end of each line with 'text'
    sed 's/test/text/2g' file.txt
    #
    # replace the 'test' in the 2nd line with 'text'
    sed '2s/test/text/g' file.txt

    2>/dev/null 1>&2

    https://unix.stackexchange.com/questions/163352/what-does-dev-null-21-mean-in-this-article-of-crontab-basics
    https://stackoverflow.com/questions/818255/in-the-shell-what-does-21-mean

–

https://unix.stackexchange.com/questions/11376/what-does-double-dash-mean

switch to root user

su –> enter password of root user

packaging and unzip (extraction)

  • unzip ：$ tar zxvf fileName.tar.gz
    For .gz file, it may appears as:
    1
    2
    3
    4
    $ tar zxvf loggd_persistent.log.0.gz
    tar: This does not look like a tar archive
    tar: Skipping to next header
    tar: Exiting with failure status due to previous errors
    Then use:
    1
    2
    3
    $ gzip -d <source-file>
    # or
    $ gunzip <source-file>
  • packaging: tar zcvf fileName.tar.gz dirPath
    or use
    1
    2
    3
    $ gzip <source-file>                      # will not keep the source file
    # or
    $ gzip -c <source-file> > <zipped-file>   # will keep both source file and zipped file
    To zip a folder:
    1
    $ gzip -r <directory>

    file type

    for details, see https://blog.csdn.net/rong09_13/article/details/79233956
    1
    $ file <fileName>

    search history cmd

    use ctrl + r to search history input, continue input until find the target cmd, then press Enter to execute.

Port

  1. check port status
    1
    lsof -i:<port-id>
    e.g.
    1
    2
    3
    [root@host63 ~]# lsof -i:32228
    COMMAND     PID USER   FD   TYPE    DEVICE SIZE/OFF NODE NAME
    kube-prox 21592 root   16u  IPv6 171971606      0t0  TCP *:32228 (LISTEN)
  2. check the process running on certain port
    1
    netstat -tunlp | grep <port-id>
    e.g.
    1
    2
    [root@host63 ~]# netstat -tunlp | grep 32228
    tcp6       0      0 :::32228                :::*                    LISTEN      21592/kube-proxy

    search for files

    1
    2
    $ find / -name *<tap here>* # search from all files
    $ find . -name *<tap here>* # search under current path
    or
    1
    $ whereis <name here>

    whereis命令只能用于程序名的搜索，而且只搜索二进制文件（参数-b）、man说明文件（参数-m）和源代码文件（参数-s）。如果省略参数，则返回所有信息

For more. refer http://www.ruanyifeng.com/blog/2009/10/5_ways_to_search_for_files_using_the_terminal.html

permit

  1. give execution permit to certain file:
    1
    $ chmod +x file.sh
    Then file.sh is a executable file and display in green, also use - can revert permit and the target file will return back to white.
    1
    $ chmod -x file.sh
  2. 777: give read, write and execution permit to file/folder:
    1
    $ chmod -R 777 *
    Above cmd means give permits to all files or folders under current path, also we can use specific file name to institute *. \

    777:分别对应文件实际拥有者，文件实际拥有者所在的组，其它用户的权限，数字权限是基于八进制数字系统而创建的，读权限（read，r）的值是4，写权限（write，w）的值是2，执行权限（execute，x）的值是1，没有授权的值是0

tcsh set resource

Linux usually use bash as the default shell, but for specific working env whcih only tcsh is allowed, use the following file to change the shell style. It serves as .bashrc

1
$ vi ~/.cshrc

tcsh is an improved version of csh (c-shell)

list files/folders by timestamp (asc/desc)

1
2
$ ll -t       # list files by time desc
$ ll -t |tar  # by asc

or

1
$ ls -lart    # desc

-bash: cannot create temp file for here-document: No space left on device

when using tab on linux and got the above error, you should check if the disk usage on current host is full.

1
2
$ df -h
$ du -sh /*   # check which folder use the most memory

ref: https://blog.csdn.net/muriyue6/article/details/84783107

vim show line number

  1. click esc and :
  2. input set nu

vimdiff - compare file diff on linux

1
$ vimdiff <file1> <file2>

:q twice to quit the comparison