Tuesday, 20 October 2009

pv or pipe viewer — one of the most useful UNIX commands

Have you ever heard about UNIX pv command? I bet you didn’t — and I promise you, your life will change and you’ll become warm and fuzzy inside — because pv stands for “pipe viewer”.

Try to count, just how many times did you do something like:

grep -i something file.txt | gzip -c > output.gz

and wondering, just how the hell long will it take for it to finish? Well, for me it did happen quite often, especially when handling 10GB+ files (such as, say, Wikipedia dumps). That is exactly when pipe viewer comes to the scene. That’s how it looks:

[user@host wikipedia]$ pv enwiki.xml | wc -l
767MB 0:00:16 [67.9MB/s] [> ] 3% ETA 0:07:58

It’s the most simple use case — but so far one of the most useful ones (if you can say that about the use case). Say, you pipe more than one command and want to display the progress separately for every of them — no problems, you can do it! — as well as many other things.

Check out the pv’s webpage: http://www.ivarch.com/programs/pv.shtml — to me it’s one of the little utilities which make my life substantially easier.

No comments:

Post a Comment