Iterating through a file

[Belisarius]

I want to iterate through the lines in a text file (they aren't single-word lines). What's the best way to do that in a shell script?

[sde]

do you have a php interpretor?

PHP Code:

#! /usr/bin/php4
<?php
// myscript.php

$array = file('/home/me/myfile.txt');

if(is_array($array)){

  foreach($array as $line){
    // do something with line
  }
}
?>

$ chmod u+x myscript.php
$ ./myscript.php

i like using php as a server-side scripting language. =)

[Belisarius]

I thought about it, but I'd need to install PHP. I was hoping there was a way to do it with regular Unix tools, but messing around with wc, head and tail just is too ugly. . .

[idx]

Here's a snippet from a script I use to remove users from AIX in bulk.. There's probably a better way, but I had to scrap something together without much fuss..

for i in `cat $1|grep -v -E '^#'|cut -d"|" -f2`
do
   if [ $2 = "-k" ]; then
      echo "Saving lsuser info - '$i'"
      lsuser -f $i |tee -a $1.lsuser
      echo "--------------------------" >> $1.lsuser
      echo "Removing user - '$i'"
      rmuser -p $i
   else
      echo "Skipping user - '$i'"
   fi

done

So each line in the loop is stored in $i .. Not sure if you need to split the line or get certain pieces, but that may help..

(worst case use perl.. it'll probably be installed)

-r

[idx]

Oh yeah, my input file was pipe delimited, so the cut -d"|" -f2 is just grabbing the username.. Perhaps adjust the entire line to `cat $1` for no filtering.

-r

[teknomage1]

I think if you ned to do arbitrary transforms to lines, sed is the best utility. If you only need some lines grep is the ticket.

[Belisarius]

I'm transforming some HTML output I grab with CURL into a csv. I'm 99% of the way there (with a number of sed calls), but the problem is I can't simply walk through line-by-line to attach the <td> values together. I guess I'll play around with sed some more, and transform </td> into a comma and try to remove the CR/LF, eliminating the need to walk through.

[teknomage1]

if sed runs out of steam, the next step would be perl with the HTML::Treebuilder module.
Alternatively 'perl -pe 's/regex/subst/g' is a great sed substitute. 'perl -pe 's/\n//mg' will turn a file with unix line endings into one really long line for example.

[Belisarius]

I can't figure out how to insert a newline in sed. \n only gives me an "n". *sigh* - that was the last hurdle. I think I'm going to switch over to PHP because I really don't want to learn Perl just for this.

[fp_unit]

#!/usr/bin/perl

open(FILE, "> yourfile.txt");
while(line = <FILE>)
{
    # do something with line
}

[teknomage1]

The reason you can't do that with sed, is because sed only sees one line at a time. But like I said, perl can be a good sed substitute, if invoked as perl -pe 'your old sed directives' it behaves exactly like sed. The addition of m to a search makes it a multiline search, which sed can't usually do. Also if your sed scripts are huge, you can uses s2p to have it automatically converted to a perl script (though it'll be even less legible that hand written perl).

[Belisarius]

Actually, I got it working using "tr" to strip out new lines, and turning the whole thing into a single line. From there I turned </tr> into a newline (had to install GNU sed to get it to work - the sed that was on Solaris9 really disliked my attempts to insert a newline), and </td><td> into a comma. Seems to have worked, I'll test it out tomorrow and see if the program that reads the CSV likes it.

[idx]

Originally Posted by fp_unit

#!/usr/bin/perl

open(FILE, "> yourfile.txt");
while(line = <FILE>)
{
    # do something with line
}

? No, that would write to the file.

-r