🏃‍♂️ Usage

puniq

🗣 Description

Delete duplicated lines.

GNU core utilities have uniq command for deleting duplicate lines. However, uniq command deletes only continuous duplicate lines. When deleting not continuous duplicate lines, we use sort command together, in that case, the order of the list was not kept.

We want to delete not continuous duplicated lines with remaining the order. This command is the same as uniq2

🏃‍♂️ Usage

puniq [OPTIONS] [INPUT [OUTPUT]]
OPTIONS
    -a, --adjacent        delete only adjacent duplicated lines.
    -d, --delete-lines    only prints deleted lines.
    -i, --ignore-case     case sensitive.

    -h, --help            print this message.
INPUT       gives file name of input.  If argument is single dash ('-')
            or absent, the program read strings from stdin.
OUTPUT      represents the destination.

🚴‍♂️ Demo

$ cat -n testdata/test1.txt
1	a1
2	a1  # <- is the duplicate of the previous line.
3	a2
4	a2  # <- is the duplicate of the previous line.
5	a3
6	a4
7	a1  # <- is the duplicate of the first line.
8	A1
$ puniq testdata/test1.txt
a1
a2
a3
a4
A1
$ puniq -a testdata/test1.txt  # same result as uniq command.
a1
a2
a3
a4
a1  # <- this line is not deleted.
A1
$ puniq -i testdata/test1.txt # ignore case
a1
a2
a3
a4
$ puniq -d testdata/test1.txt # print delete lines.
a1
a2
a1

🏄‍♂️ Utility

Delete duplicate entries in PATH

export PATH=$(echo $PATH | tr : '\n' | puniq | paste -s -d : -)

ptake

🗣 Description

take lines while satisfying the predicate.

🏃‍♂️ Usage

ptake [OPTIONS] [FILEs...]
OPTIONS
    -b, --bytes <NUMBER>       take NUMBER bytes (same as head command).
    -n, --lines <NUMBER>       take NUMBER lines (same as head command).
    -u, --until <KEYWORD>      take lines until KEYWORD is appeared.
    -w, --while <PREDICATE>    take lines while PREDICATE is satisfied.
                               we can use the variable PLINE and PLINECOUNT
                               which are the current line and its line number in the PREDICATE.
    -q, --no-header            suppress printing of headers when multiple files are being examined.

    -h, --help                 print this message and exit.
FILE
    gives file name for the input. if this argument is single dash ("-") or absent,
    it reads strings from STDIN.
    if more than a single file is specified, each file is separated by a header
    consisting of the string "==> XXX <==" where "XXX" is the name of the file.

pskip

🗣 Description

skip lines while satisfying the predicate.

🏃‍♂️ Usage

skip [OPTIONS] [FILEs...]
OPTIONS
    -b, --bytes <NUMBER>       skip NUMBER bytes (same as head command).
    -n, --lines <NUMBER>       skip NUMBER lines (same as head command).
    -u, --until <KEYWORD>      skip lines until KEYWORD is appeared.
    -w, --while <PREDICATE>    skip lines while PREDICATE is satisfied.
                               we can use the variable PLINE and PLINECOUNT
                               which are the current line and its line number in the PREDICATE.
    -q, --no-header            suppress printing of headers when multiple files are being examined.

    -h, --help                 print this message and exit.
FILE
    gives file name for the input. if this argument is single dash ("-") or absent,
    it reads strings from STDIN.
    if more than a single file is specified, each file is separated by a header
    consisting of the string "==> XXX <==" where "XXX" is the name of the file.

ptest

🗣 Description

another implementation of test command for evaluating the predicate of pskip and ptake.

🏃‍♂️ Usage

ptest <expression>
file operation
    -b file            true if file exists and is a block special file.
    -c file            true if file exists and is a character special file.
    -d file            true if file exists and is a directory.
    -e file            true if file exists (regardless of type).
    -f file            true if file exists and is a regular file.
    -g file            true if file exists and its set group ID flag is set.
    -k file            true if file exists and its sticky bit is set.
    -p file            true if file is a named pipe (FIFO).
    -r file            true if file exists and is readable.
    -s file            true if file exists and has a size greater than zero.
    -t file_descriptor true if the file whose file descriptor number is file_descriptor is open and is
                       associated with a terminal.
    -u file            true if file exists and its set user id flag is set.
    -w file            true if file exists and is writable.
    -x file            true if file exists and is executable.  If file is directory,
                       true indicates that file can be searched.
    -L file            true if file exists and its a symbolic link.
    -O file            true if file exists and its owner matches the effective user id of this process (not support on Windows platform).
    -G file            true if file exists and its group matches the effective group id of this process (not support on Windows platform).
    -S file            true if file exists and its a socket.
    file1 -nt file2    true if file1 exists and is newer than file2.
    file1 -ot file2    true if file1 exists and is older than file2.
    file1 -ef file2    true if file1 and file2 exist and refer to the same file.
string operation
    -n string          true if the length of string is nonzero.
    -z string          true if the length of string is zero.
    string             true if string is not the null string.
    s1 = s2            true if the strings s1 and s2 are identical.
    s1 == s2           true if the strings s1 and s2 are identical, same as s1 = s2.
    s1 != s2           true if the strings s1 and s2 are not identical.
    s1 -starts s2      true if string s1 starts with s2.
    s1 -ends s2        true if string s1 ends with s2.
    s1 -contains s2    true if string s1 contains s2.
    s1 < s2            true if string s1 comes before s2 based on the binary value of their characters.
    s1 <= s2           true if string s1 comes before or equals to s2 based on the binary value of their characters.
    s1 > s2            true if string s1 comes after s2 based on the binary value of their characters.
    s1 >= s2           true if string s1 comes after or equals to s2 based on the binary value of their characters.
numeric operation
    n1 -eq n2          true if the integers n1 and n2 are algebraically equal.
    n1 -ne n2          true if the integers n1 and n2 are not algebraically equal.
    n1 -gt n2          true if the integers n1 is algebraically greater than the integer n2.
    n1 -ge n2          true if the integers n1 is algebraically greater than or equals to the integer n2.
    n1 -lt n2          true if the integers n1 is algebraically less than the integer n2.
    n1 -le n2          true if the integers n1 is algebraically less than or equals to the integer n2.
combined operation
    ! expression                true if the expression is false.
    expression -a expression    true if both expression1 and expression2 are true.  The -a operator has higher precedence than the -o operator.
    expression -o expression    true if either expression1 or expression2 are true.  The -o operator has lower precedence than the -a operator.
    ( expression )              true if expression is true

snip

🗣 Description

snip command is synthesis of head and tail command.

🏃‍♂️ Usage

snip [OPTIONS] [FILEs...]
OPTIONS
  -H, --head int       print first HEAD lines (same as head command). (default -1)
  -T, --tail int       print last TAIL lines (same as tail command). (default -1)
  -N, --number int     print first and last lines (default is 5). (default 5)
  -n, --line-number    print line number with output lines.
  -s, --no-snip-sign   suppress printing of snip sign and the number of snipped lines.
  -q, --no-header      suppress printing of headers when multiple files are being examined.
  -h, --help           print this message and exit
  -v, --version        print the version information and exit

FILE
  gives file name for the input. if this argument is single dash ('-') or absent,
  it reads strings from STDIN.
  if more than a single file is specified, each file is separated by a header
  consisting of the string '==> XXX <==' where 'XXX' is the name of the file.

🚴‍♂️ Demo