think tank forum

technology » Unix and Unix-like CLI

DaGr8Gatzby's avatar
15 years ago
link
DaGr8Gatzby
Drunk by Myself
Here. We need this.
maple's avatar
15 years ago
link
maple
i like large datasets
i dont get it.
asemisldkfj's avatar
15 years ago
link
asemisldkfj
the law is no protection
everyone was hating on the Windows command line :).

what should we talk about in here?
lucas's avatar
15 years ago
link
lucas
i ❤ demo
related threads:
shell script heads
Random Scripts and Stuff
maple's avatar
15 years ago
link
maple
i like large datasets
ksh ftw!
bluet's avatar
15 years ago
link
bluet
> ksh ftw!

!!
asemisldkfj's avatar
15 years ago
link
asemisldkfj
the law is no protection
ksh rules! zsh is not bad either, mostly for the nice tab-completion stuff.
maple's avatar
15 years ago
link
maple
i like large datasets
side note: i've been bored and have this Thinkpad X41T that I always have OpenBSD on but never use full time (use a MBP normally). Been doing an experiment to see if I could actually use it as a dev environment and desktop full time and recording all the things I did to get it there... should be up soon. figured you bsd nerds might be interested.
lucas's avatar
15 years ago
link
lucas
i ❤ demo
ksh is a great openbsd native shell.
maple's avatar
15 years ago
link
maple
i like large datasets
what do you mean lucas? all shells are native to every os they run on.

oh an 'set -o vi' is the best thing ever invented.

works in bash, ksh, zsh and more im sure
lucas's avatar
15 years ago
link
lucas
i ❤ demo
the ksh on openbsd is different from the ksh included with other oses.
maple's avatar
15 years ago
link
maple
i like large datasets
thats absolutely true: http://forums.bsdnexus.com/viewtopic.php?id=1265
nny's avatar
15 years ago
link
nny
M̮͈̣̙̰̝̃̿̎̍ͬa͉̭̥͓ț̘ͯ̈́t̬̻͖̰̞͎ͤ̇ ̈̚J̹͎̿̾ȏ̞̫͈y̭̺ͭc̦̹̟̦̭̫͊̿ͩeͥ̌̾̓ͨ
vimperator for firefox... AWESOME
asemisldkfj's avatar
15 years ago
link
asemisldkfj
the law is no protection
I remember wishing that existed like two years ago. I am glad it does now.
maple's avatar
15 years ago
link
maple
i like large datasets
side note. flash is workin in openbsd these days nicely. it used to suck ass but now its good. i only use flash mainly for finding new music and occasional youtube, but its really nice to have the option.
DaGr8Gatzby's avatar
15 years ago
link
DaGr8Gatzby
Drunk by Myself
ZSH :)
asemisldkfj's avatar
15 years ago
link
asemisldkfj
the law is no protection
question, maple: how do you have it working? what browser and what flash port/package? flash is one thing that kind of kills the idea of a bsd desktop for me because I remember how much of a pain it was a couple of years ago.
maple's avatar
15 years ago
link
maple
i like large datasets
asemi,

thats been the case for me too. i dont really like flash but it i use it too much (youtube, alot of music sites, etc).

anyway there are two ways to do it. opera+flash or firefox+flash.

opera+flash both run under linux emulation. for this just run:

# cd /usr/ports/www/opera-flashplugin
# make && make install

that will build opera, flash and setup the fedora-base.

but what i've been running is firefox3+flash7. with this setup firefox is a native openbsd build but the flash7 is running under linux emul.

im running a snapshot of openbsd 4.5 (4.4 has firefox 2.x) on this machine.

sorry for the long link -- its cached as the site seems to be down:

http://74.125.77.132/search?q=cache :tAT7yu7vA94J:openbsd.stanleylieber.com/txt/openbsd_firefox_flash_player_7.txt+openbsd+firefox+flash+player+7&hl=en&ct=clnk&cd=8&gl=us&client=firefox-a


maple's avatar
15 years ago
link
maple
i like large datasets
well F that link too -- here http://www.dugoo.net/wordpress/?p=217
asemisldkfj's avatar
15 years ago
link
asemisldkfj
the law is no protection
I'm trying this now!
asemisldkfj's avatar
15 years ago
link
asemisldkfj
the law is no protection
yep, totally didn't work. I gave up when the nspluginwrapper package wouldn't build.
DaGr8Gatzby's avatar
15 years ago
link
DaGr8Gatzby
Drunk by Myself
Here is a neat trick I learned today. On compromised servers you will usually see a file that has a name that you can't really edit with the shell(\ ? & etc.). Here is something easy I picked up:

# To rename shitty named files, grab the inode number with the first command. Then make sure to use the 2nd command to rename it. The {} is the file name found by find. The \ prevents expansion and the ; terminates the statement:

ls -hali

find ./ -inum INODE -exec COMMAND {} \;

asemisldkfj's avatar
15 years ago
link
asemisldkfj
the law is no protection
cool!
nny's avatar
15 years ago
link
nny
M̮͈̣̙̰̝̃̿̎̍ͬa͉̭̥͓ț̘ͯ̈́t̬̻͖̰̞͎ͤ̇ ̈̚J̹͎̿̾ȏ̞̫͈y̭̺ͭc̦̹̟̦̭̫͊̿ͩeͥ̌̾̓ͨ
That'd also be good for network shares to other filesystems...
maple's avatar
15 years ago
link
maple
i like large datasets
posted this link in another thread, but incase someone reads this in the future

openbsd+flash = http://fakeempire.com/posts/firefox-flash-vim … r-openbsd/
asemisldkfj's avatar
15 years ago
link
asemisldkfj
the law is no protection
gnash is working pretty great for me lately. most website that use it for navigation and stuff work perfectly. youtube is hit or miss but works sometimes.
andre's avatar
15 years ago
link
andre
Gatz, that's cool... Sometimes what I do when I don't want to figure out all the escaping is to use some programming language to rename the file without going through the shell special character stuff. For exameple, File.mv('&?\', 'stupid_file'), in Ruby.
DaGr8Gatzby's avatar
15 years ago
r3, link
DaGr8Gatzby
Drunk by Myself
^ Andre, I just saw this. Wow. That is pretty nifty. I do like to use the built in tools for the OS. Those people thought of everything!

Here is one trick I feel I should share. I was on a customer's server and I noticed that there were some viruses on it. So of course, I use clamscan to get the detected files out from the log:

root@server [~]# grep FOUND clamscan.log | grep mail | awk {'print $1'}
/home/sw1mm1ng/mail/swimmingpooldehumidifiers.org/craigd/cur/1257842183.H61105P30680.server.virtualassistant.org,S=42582:2,S:
/home/sw1mm1ng/mail/swimmingpooldehumidifiers.org/craigd/cur/1257754383.H617595P8760.server.virtualassistant.org,S=31079:2,S:
/home/sw1mm1ng/mail/swimmingpooldehumidifiers.org/craigd/cur/1257832532.H374077P7133.server.virtualassistant.org,S=42571:2,S:
/home/sw1mm1ng/mail/swimmingpooldehumidifiers.org/craigd/cur/1257794501.H93549P13514.server.virtualassistant.org,S=42599:2,S:
/home/sw1mm1ng/mail/swimmingpooldehumidifiers.org/craigd/cur/1257812246.H821056P24563.server.virtualassistant.org,S=42577:2,S:


Oh no! There seems to be a colon at the end! I can't pipe this to xargs or write a loop, since it will complain the file is not there. What to do? Enter ... rev.

A simple cut will not work, since it works on the length of the string. So here is what you do:

root@server [~]# grep FOUND clamscan.log | grep mail | awk {'print $1'} | rev | cut -b 2- | rev
/home/sw1mm1ng/mail/swimmingpooldehumidifiers.org/craigd/cur/1257842183.H61105P30680.server.virtualassistant.org,S=42582:2,S
/home/sw1mm1ng/mail/swimmingpooldehumidifiers.org/craigd/cur/1257754383.H617595P8760.server.virtualassistant.org,S=31079:2,S
/home/sw1mm1ng/mail/swimmingpooldehumidifiers.org/craigd/cur/1257832532.H374077P7133.server.virtualassistant.org,S=42571:2,S
/home/sw1mm1ng/mail/swimmingpooldehumidifiers.org/craigd/cur/1257794501.H93549P13514.server.virtualassistant.org,S=42599:2,S
/home/sw1mm1ng/mail/swimmingpooldehumidifiers.org/craigd/cur/1257812246.H821056P24563.server.virtualassistant.org,S=42577:2,S


The command is doing the following:

Filtering for all found output | filtering the filtered output for mail(there were some false positives) | only return the first column(I don't need found) | reverses the line | cuts the first 2 characters of the line(there is a \n so 2 are required and the - removes) | reverse the reversed output

And here are the results:

root@server [~]# grep FOUND clamscan.log | grep mail | awk {'print $1'} | rev | cut -b 2- | rev | xargs rm -fv
removed `/home/sw1mm1ng/mail/swimmingpooldehumidifiers.org/craigd/cur/1257842183.H61105P30680.server.virtualassistant.org,S=42582:2,S'
removed `/home/sw1mm1ng/mail/swimmingpooldehumidifiers.org/craigd/cur/1257754383.H617595P8760.server.virtualassistant.org,S=31079:2,S'
removed `/home/sw1mm1ng/mail/swimmingpooldehumidifiers.org/craigd/cur/1257832532.H374077P7133.server.virtualassistant.org,S=42571:2,S'
removed `/home/sw1mm1ng/mail/swimmingpooldehumidifiers.org/craigd/cur/1257794501.H93549P13514.server.virtualassistant.org,S=42599:2,S'
removed `/home/sw1mm1ng/mail/swimmingpooldehumidifiers.org/craigd/cur/1257812246.H821056P24563.server.virtualassistant.org,S=42577:2,S'



Pretty nice huh? I know it was only for five files here. But typically you have ~100-~1000 of these lines. This little tidbit will save you an enormous amount of time.
 
15 years ago
r3, link
arun
keep smiling !
Gatz,

$ awk -F ':' '/mail.*FOUND$/ { print $1FS$2 }' clamscan.log | xargs rm -fv
andre's avatar
15 years ago
r3, link
andre
Also, a simple cut does work:

$ grep "mail.*FOUND" clamscan.log | awk {'print $1'} | cut -d: -f-2
DaGr8Gatzby's avatar
15 years ago
link
DaGr8Gatzby
Drunk by Myself
Yea but you guys weren't online when I needed this.

dannyp's avatar
15 years ago
link
dannyp
dʎuuɐp
andre, arun, your options work but it's hardly as cool as tossing a couple reverses on the line :D

anyway thanks for this thread!
 
15 years ago
link
arun
keep smiling !
I like the way gatz's brain worked here. I would've never thought about it that way.
DaGr8Gatzby's avatar
14 years ago
link
DaGr8Gatzby
Drunk by Myself
Resurrecting this because my find command was not working. There was a suspicious file in /. When I tried searching, it took an absurd amount of time. I have resolved it with the following:

# find / -xdev -inum INODE_NUM -ok mv '{}' suspicious.txt \;

-ok will prompt you. If you want to skip the prompt, use -exec.
DaGr8Gatzby's avatar
12 years ago
link
DaGr8Gatzby
Drunk by Myself
Fuck this shit is killing me. I have a huge FTP site and I need to download certain files with a bash script. The FTP program supports globbing. However, I can't get the glob working. Here is the pattern I am using:

firstRegex = XNYM_CL_CL
mls $firstRegex* -

output
-------------------------------------

XNYM_CL_CL_FUT_20120426.zip
XNYM_CL_CL_FUT_20120427.zip
XNYM_CL_CL_FUT_20120428.zip
XNYM_CL_CL_FUT_20120429.zip
XNYM_CL_CL_FUT_20120430.zip

mls $firstRegex*_$year* -

output
------------------------------------
XNYM_CL_CL_FUT_20120506.zip
XNYM_CL_CL_FUT_20120507.zip
XNYM_CL_CL_FUT_20120508.zip
XNYM_CL_CL_FUT_20120509.zip
XNYM_CL_CL_FUT_20120510.zip

Why does it not pick up the 04 files?
DaGr8Gatzby's avatar
12 years ago
link
DaGr8Gatzby
Drunk by Myself
WTF shit just starts working?!?!
asemisldkfj's avatar
12 years ago
link
asemisldkfj
the law is no protection
hahahaha
DaGr8Gatzby's avatar
12 years ago
link
DaGr8Gatzby
Drunk by Myself
This was NOT fun.
DaGr8Gatzby's avatar
12 years ago
link
DaGr8Gatzby
Drunk by Myself
This page saved me today:

http://www.cyberciti.biz/tips/linux-unix-get- … -date.html
DaGr8Gatzby's avatar
12 years ago
link
DaGr8Gatzby
Drunk by Myself
I literally lost 4 hours today because I did not know the paste command. WTF ... :(
DaGr8Gatzby's avatar
12 years ago
link
DaGr8Gatzby
Drunk by Myself
You can put FTP commands in a batch file and redirect them to FTP. Pretty sweet:

# ftp HOST < batch_file.txt
DaGr8Gatzby's avatar
12 years ago
r1, link
DaGr8Gatzby
Drunk by Myself
Never really was much of a shell junkie. Turns out {#..#} works great when you need to filter based on range:

$ du -chs {201204{23..30},201205*,201206*}
2.5G    20120423
2.4G    20120424
2.9G    20120425
2.5G    20120426
2.4G    20120427
311M    20120428
17M     20120429
2.1G    20120430
2.0G    20120501
2.5G    20120502
2.5G    20120503
2.8G    20120504
22M     20120505
17M     20120506
2.3G    20120507
2.8G    20120508
3.0G    20120509
2.8G    20120510
2.5G    20120511
105M    20120512
17M     20120513
2.5G    20120514
2.9G    20120515
3.5G    20120516
3.4G    20120517
3.4G    20120518
255M    20120519
17M     20120520
3.0G    20120521
3.2G    20120522
3.9G    20120523
3.6G    20120524
2.9G    20120525
16M     20120527
553M    20120528
3.2G    20120529
3.5G    20120530
3.6G    20120531
4.1G    20120601
191M    20120602
17M     20120603
3.4G    20120604
3.1G    20120605
3.3G    20120606
3.8G    20120607
3.3G    20120608
22M     20120609
17M     20120610
183M    20120611
102G    total


Edit: Years of TTF, always search for the guide.
asemisldkfj's avatar
12 years ago
link
asemisldkfj
the law is no protection
you couldn't remember pre tags?
asemisldkfj's avatar
12 years ago
link
asemisldkfj
the law is no protection
oh, html brackets :).
DaGr8Gatzby's avatar
12 years ago
link
DaGr8Gatzby
Drunk by Myself
Very very helpful:

|| in a shell script means that if the first command's return code is not 0, execute the second command.

Example:

$ command -v fo0 >/dev/null 2>&1 || echo >&2 "Command not found"
Command not found

An existing command is blank. :)
Carpetsmoker's avatar
12 years ago
link
Carpetsmoker
Martin
|| is "or", like most programming languages.

I strongly prefer to check exit codes with $?, it may be a few more lines, but IMHO that's always better than squeezing multiple statements on a single line. It also allows you to do better extended error handling or add it later.

I also prefer to just show the error message, now you redirect everything from "command" to /dev/null and just *assume* that *any* error you encounter mean that "command" doesn't exist. That's just silly. I've had to hack many, MANY, *MANY* shell script which make these bad assumptions just to get a meaningful error message...

The *proper* procedure is to check if the command exists with which(1), execute the command and capture the output in a variable, check if $? is non-0 (usually, may be different for some tools, like grep), display error output from variable if it isn't.
DaGr8Gatzby's avatar
12 years ago
link
DaGr8Gatzby
Drunk by Myself
Please don't say which is the proper way. That really isn't posix compliant.

I agree with the $? operator. That is a pretty neat trick I also came across today.
asemisldkfj's avatar
11 years ago
r2, link
asemisldkfj
the law is no protection
I spent a looooot of time at the command line today. here's the script I'm about to set up as a cron job on my pf firewall to dynamically update my IP address: http://code.google.com/p/gandi-automatic-dns/

in short, it constructs a few XML RPCs given a method name and datatype/value pairs, pipes the XML (preceded by a POST) to openssl s_client, and talks to my domain registrar's API. I initially tried to do all of this in as few RPCs as possible because I just wanted to git r done but I finally gave up and wrote a function for them which cut the size of the script by more than half.

XML sucks.
asemisldkfj's avatar
11 years ago
link
asemisldkfj
the law is no protection
oh and handling arguments with while loops and shift was fun. hadn't done that before.
phi_'s avatar
11 years ago
r1, link
phi_
... and let the Earth be silent after ye.
Yeah, I'm having issues myself with some shell script for dwm's status bar on FreeBSD.

It was easy enough on linux to just use /sys/blah/blah/BAT0 for battery information, but FreeBSD don't like that one bit, so currently I'm using a very, very ugly bit of code to parse acpiconf. I KNOW there's a simpler way to do it. Anyone have a suggestion? I feel like a noob right now. :(

acpiconf -i 0 | grep -n : | grep '14:' | awk '{print $3}'


acpiconf -i 0 dumps information about the battery, I need the 14th line (percent remaining, hence the greps), but only the percentage part (hence the awk).
asemisldkfj's avatar
11 years ago
link
asemisldkfj
the law is no protection
post the output of acpiconf -i 0. there's no command that will return more specific information? I remember being able to use the apm command on openbsd.
asemisldkfj's avatar
11 years ago
link
asemisldkfj
the law is no protection
but I guess if you're using acpi...
Carpetsmoker's avatar
11 years ago
r2, link
Carpetsmoker
Martin
The apm command won't work on FreeBSD 64-bit.

AFAIK acpiconf is the way to go, that's also how I do it (1)
You could use cut(1) instead of awk(1) though ;)

1: https://bitbucket.org/Carpetsmoker/battray/sr … at=default

PS: How do I add links on ttf lucas? You should add a cheatsheet at the bottom ...
asemisldkfj's avatar
11 years ago
link
asemisldkfj
the law is no protection
you search for 'guide' and then you click on a result and then you realize that it's a thread someone created by accident when searching for a guide and then you go back and click on the real guide thread.
phi_'s avatar
11 years ago
link
phi_
... and let the Earth be silent after ye.
Here's the output:

$ acpiconf -i 0
Design capacity:        6600 mAh
Last full capacity:     6511 mAh
Technology:             secondary (rechargeable)
Design voltage:         10800 mV
Capacity (warn):        512 mAh
Capacity (low):         256 mAh
Low/warn granularity:   66 mAh
Warn/full granularity:  66 mAh
Model number:           1005HA
Serial number:           
Type:                   LION
OEM info:               ASUS
State:                  high 
Remaining capacity:     100%
Remaining time:         unknown
Present rate:           0 mA (0 mW)
Present voltage:        12577 mV


CS: thanks!
asemisldkfj's avatar
11 years ago
link
asemisldkfj
the law is no protection
will someone please tell me why this:

sed -n 's/R.*c.*\([0-9]*%\)/\1/p'


echos only the % sign?
bluet's avatar
11 years ago
r8, link
bluet
Your last .* is too greedy. It consumes all characters between c and %.

Adding a space will make it work:

sed -n 's/R.*c.* \([0-9]*%\)/\1/p'


But I think this is clearer:

sed -n 's/Remaining capacity: \+\([0-9]\+%\)/\1/p'


However, none of those will actually work, since acpiconf uses tabs ( http://svnweb.freebsd.org/base/release/9.1.0/ … iew=markup ):

if (battio.battinfo.cap == -1)
        printf("Remaining capacity:\tunknown\n");
else
        printf("Remaining capacity:\t%d%%\n",
            battio.battinfo.cap);


This will work:

sed -n 's/Remaining capacity:\t\([0-9]\+%\)/\1/p'
asemisldkfj's avatar
11 years ago
link
asemisldkfj
the law is no protection
ah of course. damn greedy asterisk. sed is fun. the re_format man page is also a good read.
 
11 years ago
link
arun
keep smiling !
$ acpiconf -i 0 |  awk '/Remaining capacity/ { print $NF }'
phi_'s avatar
11 years ago
link
phi_
... and let the Earth be silent after ye.
... this makes much more sense.
asemisldkfj's avatar
11 years ago
link
asemisldkfj
the law is no protection
but, sed is more fun!
DaGr8Gatzby's avatar
11 years ago
link
DaGr8Gatzby
Drunk by Myself
It's time for bed ... To Bed! I sed!
Carpetsmoker's avatar
11 years ago
link
Carpetsmoker
Martin
World-class joke! have you considered a career in comedy?
DaGr8Gatzby's avatar
11 years ago
link
DaGr8Gatzby
Drunk by Myself
The only comedy for me is this joke of a life.
asemisldkfj's avatar
11 years ago
link
asemisldkfj
the law is no protection
life is pretty hilarious.
Carpetsmoker's avatar
11 years ago
r1, link
Carpetsmoker
Martin
It's not unlike a slapstick humor where the main characters keep falling on their face no matter what they do.
nny's avatar
11 years ago
link
nny
M̮͈̣̙̰̝̃̿̎̍ͬa͉̭̥͓ț̘ͯ̈́t̬̻͖̰̞͎ͤ̇ ̈̚J̹͎̿̾ȏ̞̫͈y̭̺ͭc̦̹̟̦̭̫͊̿ͩeͥ̌̾̓ͨ
http://www.music-piracy.com/?p=804
lucas's avatar
11 years ago
r1, link
lucas
i ❤ demo
trying to decipher those boxes

whoa, i put them in this post, but they disappeared. :[
nny's avatar
11 years ago
link
nny
M̮͈̣̙̰̝̃̿̎̍ͬa͉̭̥͓ț̘ͯ̈́t̬̻͖̰̞͎ͤ̇ ̈̚J̹͎̿̾ȏ̞̫͈y̭̺ͭc̦̹̟̦̭̫͊̿ͩeͥ̌̾̓ͨ
The results... screenshotted from ubuntu:

http://www.flickr.com/photos/openfly/8618771538/
asemisldkfj's avatar
11 years ago
link
asemisldkfj
the law is no protection
that is excellent. I need to put return codes in my prompt!
lucas's avatar
11 years ago
link
lucas
i ❤ demo
pretty cool. i don't know about your user title, though. :p
Carpetsmoker's avatar
11 years ago
link
Carpetsmoker
Martin
tcsh has the `printexitvalue' settings, which does the same thing. Albeit in a much more boring way.