Linux Format forums Forum Index Linux Format forums
Help, discussion, magazine feedback and more
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

Script Suggestions

 
Post new topic   Reply to topic    Linux Format forums Forum Index -> Programming
View previous topic :: View next topic  
Author Message
LeeNukes
LXF regular


Joined: Sun Jun 21, 2009 9:11 pm
Posts: 954
Location: At the bar

PostPosted: Fri Jan 21, 2011 10:10 pm    Post subject: Script Suggestions Reply with quote

Hi guys,

Written several bash scripts to run through a process, stuck on one due to it launching a program which it becomes stuck in.

Essentially the program runs, outputs strings into a file, once those strings have ran through, another program needs to be ran which then compares the strings which ran through a program with the strings which are in another file.

It won't skip to the next step unless the program is quit, normally Ctrl+C will do this as normal.

There is no real set time at how long the program takes, anything from 10 minutes to 30 minutes probably, but ideally I want the next step to run straight after the program finishes.

I was thinking of something with wc maybe? To compare the two files line counts, but how do I get a wc going whilst its running the program?
_________________
Join GiffGaff and get 5 free credit
Back to top
View user's profile Send private message
MartyBartfast
LXF regular


Joined: Mon Aug 22, 2005 8:25 am
Posts: 814
Location: Hants, UK

PostPosted: Fri Jan 21, 2011 10:30 pm    Post subject: Reply with quote

You could put the program into the background, and then loop round waiting for it to complete, checking it's status with jobs e.g:





Code:

$ sleep 60s &
[1] 18685
$ jobs -l     # do this in a timed loop
[1]+ 18685 Running                 sleep 60s & 
$ jobs -l
[1]+ 18685 Running                 sleep 60s &
$ jobs -l
[1]+ 18685 Running                 sleep 60s &
$ jobs -l
[1]+ 18685 Running                 sleep 60s &
$ jobs -l
[1]+ 18685 Done                    sleep 60s
$  # now go on and do the next bit

_________________
I have been touched by his noodly appendage.
Back to top
View user's profile Send private message
nelz
Site admin


Joined: Mon Apr 04, 2005 12:52 pm
Posts: 8454
Location: Warrington, UK

PostPosted: Fri Jan 21, 2011 11:19 pm    Post subject: Reply with quote

Could you make the file it writes to a FIFO? Then your script can read the data as it is being written.
_________________
"Insanity: doing the same thing over and over again and expecting different results." (Albert Einstein)
Back to top
View user's profile Send private message
LeeNukes
LXF regular


Joined: Sun Jun 21, 2009 9:11 pm
Posts: 954
Location: At the bar

PostPosted: Sat Jan 22, 2011 2:43 pm    Post subject: Reply with quote

The program by default writes to STDOUT but I'm redirecting it into a file.

Its essentially translation software, so it will take in the strings in one language and out put them to another.

It's a service though, so once its processed the file, it sits there. Waiting for input from STDIN.

As it waits for input from STDIN I don't think the loop method will work as it won't ever be "done".
_________________
Join GiffGaff and get 5 free credit
Back to top
View user's profile Send private message
nelz
Site admin


Joined: Mon Apr 04, 2005 12:52 pm
Posts: 8454
Location: Warrington, UK

PostPosted: Sat Jan 22, 2011 10:01 pm    Post subject: Reply with quote

Can you control the data it is fed, such as sending an EOF signal so the program knows there is no more data and it is time to exit.

Otherwise, you'll need to run the program in the background and read data from its file as and when it appears, possibly with a while read loop.
_________________
"Insanity: doing the same thing over and over again and expecting different results." (Albert Einstein)
Back to top
View user's profile Send private message
LeeNukes
LXF regular


Joined: Sun Jun 21, 2009 9:11 pm
Posts: 954
Location: At the bar

PostPosted: Sat Jan 22, 2011 11:35 pm    Post subject: Reply with quote

The data can be added to I guess, just as long as it doesn't try to translate it.

How would I append an end to it? Would it depend on the program supporting an EOF? Or can a ctrl+c be put in somehow?
_________________
Join GiffGaff and get 5 free credit
Back to top
View user's profile Send private message
nelz
Site admin


Joined: Mon Apr 04, 2005 12:52 pm
Posts: 8454
Location: Warrington, UK

PostPosted: Sun Jan 23, 2011 12:08 pm    Post subject: Reply with quote

That depends on the program doing the reading. Do you have any control over that?
_________________
"Insanity: doing the same thing over and over again and expecting different results." (Albert Einstein)
Back to top
View user's profile Send private message
LeeNukes
LXF regular


Joined: Sun Jun 21, 2009 9:11 pm
Posts: 954
Location: At the bar

PostPosted: Sun Jan 23, 2011 3:23 pm    Post subject: Reply with quote

nelz wrote:
That depends on the program doing the reading. Do you have any control over that?


I have control over everything, this is the line for feeding in the data at which point I get caught in the program:

Code:
# Translate scoring data using moses engine
/opt/moses/bin/moses -f $basedir/binarized/moses.ini < $scoringdir/$3.strue.$4 > $scoringdir/$3.mosesscore.$5


the -f is the config file for the particular engine, the $scoringdir/$3.strue.$4 is the input data and obviously the $scoringdir/$3.mosesscore.$5 is the output.

Here is the help for the program: http://www.leenukes.co.uk/documents/mosesinfo
_________________
Join GiffGaff and get 5 free credit
Back to top
View user's profile Send private message
nelz
Site admin


Joined: Mon Apr 04, 2005 12:52 pm
Posts: 8454
Location: Warrington, UK

PostPosted: Sun Jan 23, 2011 6:55 pm    Post subject: Reply with quote

Is the problem that moses keeps waiting for more data from $scoringdir/$3.strue.$4 even when there is no more? If the program writing to that file closed it when it had finished, I would expect moses to exit as it would see an EOF.

Or is the hangup elsewhere in the workflow?
_________________
"Insanity: doing the same thing over and over again and expecting different results." (Albert Einstein)
Back to top
View user's profile Send private message
LeeNukes
LXF regular


Joined: Sun Jun 21, 2009 9:11 pm
Posts: 954
Location: At the bar

PostPosted: Mon Jan 24, 2011 9:54 am    Post subject: Reply with quote

it just sits there waiting for more input unless you kill it, even at the end of the file.

The process is it loads its engine, then takes in the data, then when its finished, just sits there.

I'll try and craft an end of the file, see if it works.
_________________
Join GiffGaff and get 5 free credit
Back to top
View user's profile Send private message
LeeNukes
LXF regular


Joined: Sun Jun 21, 2009 9:11 pm
Posts: 954
Location: At the bar

PostPosted: Tue Jan 25, 2011 3:11 pm    Post subject: Reply with quote

Scratch this, it appears that after some time, it kills itself. Embarassed
_________________
Join GiffGaff and get 5 free credit
Back to top
View user's profile Send private message
nelz
Site admin


Joined: Mon Apr 04, 2005 12:52 pm
Posts: 8454
Location: Warrington, UK

PostPosted: Tue Jan 25, 2011 3:36 pm    Post subject: Reply with quote

Suicidal software? Shocked
_________________
"Insanity: doing the same thing over and over again and expecting different results." (Albert Einstein)
Back to top
View user's profile Send private message
View previous topic :: View next topic  
Display posts from previous:   
Post new topic   Reply to topic    Linux Format forums Forum Index -> Programming All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Linux Format forums topic RSS feed 


Powered by phpBB © 2001, 2005 phpBB Group


Copyright 2011 Future Publishing, all rights reserved.


Web hosting by UKFast