EgonDev

Some Validation

egoncasteel — Tue, 02 Mar 2010 22:33:23 +0000

I started in web programing and have found that you can never underestimate ether the users stupidity or malicious intent. That means validation on every piece of info needs to be done. In the end you will be glad you added it even thou it is annoying and boring to program.

Now there isn’t really a good reason other then the practice to add validation to the exercises you do while going through a programing book, but it just seems wrong to take input and not validate to me. So here are a couple of ways I have started doing it in Python

Pick your option

Here is a common validation task you will see a lot. The user must choose form a list of options.

print "temp conversion program"
conType = "none"
while not(conType == "ctf" or conType == "ftc"):
    conType = raw_input("cel to fahr. or fahr to cel (enter ctf or ftc): ")
sTemp = "none"

So in this chunk of code the user is being asked if he wants to go from cel to fahr. or fahr. to cel. I only want to except ctf or ftc. Any thing else will mess up the rest of the program.

Notice I set the conType var before the while loop that holds the prompt for the user. This is so the program will ask the use for input on the first time through. Now if the user inputs one of the chosses I want to see the while loop will eval false and the program will move on. If the user enters something I don’t want to see the while loop will eval true and give the user a chance to try again.

Getting the right type of input

Here is another validation problem that comes up a lot. Getting the right type of input. In this case the “type” I am talking about is data type (str, int, float, …).

change = "none"
while change == "none":
    try:
        change = float(raw_input("enter the change: "))
    except:
        print "** input must be a float **"
        change = "none"
        pass

In this code sample I am trying to get the user to give me something I can turn in to a float (ex. 12.56). Now I have still wrapped the input section in a while loop, but now we have the try and except statements in the mix. Here is what I am telling Python to do:

Try to
- Take the the users input
- Change the users input in to a float
- Assign that float
If any of the things you try raise an error
- Tell the user they failed
- Set change back to “none” (so the while loop will execute again)
- Don’t show the user any errors or stop the program
  - That is what the “pass” keyword does in the except block

This will work for any of Python’s type convert functions. With this method you get the best of both worlds. You get your input converted and you take care of validation. If you did this with a regular expression you would have all the over head of the regular expression engine, and you still wouldn’t be 100% sure if Python can convert the type because you haven’t asked it to yet .

Posting Source Code

egoncasteel — Tue, 02 Mar 2010 21:42:09 +0000

I am just tired of looking this article up so I am adding a link to it here
http://en.support.wordpress.com/code/posting-source-code/

If you cant guess it is an article on how to post code on wordpress.com blogs.

Python Modules :: Building a little Library

egoncasteel — Tue, 02 Mar 2010 21:20:50 +0000

Here is what I did for my little dev library for Python

Made a directory in my home folder called /home/egon/pyModules
Added an empty file to my home directory, the pyModules director, and inside any sub-directories named __init__.py

So what does that get you?

Well now I can import scripts buy using a statement like this:

import pyModules.easygui.easygui

This will import the easygui module from the /home/egon/pyModules/easygui directory. You can now use functions defined in the easygui module calling them as such:

pyModule.easygui.easygui.msgbox('hello')

don’t worry there is a short cut coming

Now you really don’t want to type pyModules.stuff.stuff every time you want to access a method so here is the shortcut:

egui = pyModule.easygui.easygui
egui.msgbox('hello')

See told you not to worry

Once you assign the module you imported to a new name you can use that name instead of the full reference.

So what is going on here?

Well in Ubuntu your home directory is part of your Python path (where Python gos looking when you try to import a module). By adding the __init__.py files it tells Python to treat those directories as modules. That allows us to you the dot notation to get to the module you want to import.

Why bother?

I just came up with this scheme to organize the modules I would using while learning Python. I didn’t want to just drop them into my home directory or the main Python directory. This seemed like a good way to do it to me, but I would never dare say it is a good idea for everyone. Also this maybe a really ugly way to do it. I don’t know enough about Python yet to say. Feel free to put a better solution in comments.

Back on the Snake

egoncasteel — Tue, 02 Mar 2010 20:01:47 +0000

It has been awhile, but I thought I would post some more. About 2 years ago I spent a couple of weeks learning Python, and I really liked it. Well everything I learned is now forgotten, but I thought I would take another crack at it. So you can expect to see some more Python posts here soon. Just remember I am learning Python as I go so I am sure that some of this will not be best practices. Hell some of it is going to be down right wrong I am sure.

It is almost that time of year again

egoncasteel — Tue, 13 Oct 2009 21:26:13 +0000

www.childsplaycharity.org

Awk as a Templating Language

egoncasteel — Mon, 27 Jul 2009 21:31:34 +0000

Preface

This is an article I wrote for work but I thought that it may be useful to others as well. It describes how to use awk as a simple templating language to run multiple command in a linux/unix environment. It talk specifically about a command we use called btq_submit which submits reports to our report queuing system but you could applied the technique to any comandline command.

This is a technique to use AWK to create an input and template to run reports. This method has a lot of over head so it will most likely only be useful where there is a report ran for many different sets of vars.

Overview

The command has 2 parts. The first part is the input. The input is what provides the information to the template. The template is the second part. It takes the input and writes out the btq_submit block.

#!/bin/sh

#test script for using awk to run reports

# vars

# store:[store num]:[store abbr]

# market:[market abbr]

# run line

# run:[department]:[start minor]:[end minor]

echo “

market:MSN\n

store:10:MAD\n

run:FURN:310:574Z\n

run:TV:000:046\n

run:STER:044:083Z\n

run:SE:084:149Z\n

run:COMP:150:249Z\n

run:APPL:250:309Z\n

store:12:EST\n

run:FURN:310:574Z\n

run:TV:000:046\n

run:STER:044:083Z\n

run:SE:084:149Z\n

run:COMP:150:249Z\n

run:APPL:250:309Z\n

“|awk ‘

BEGIN{

RS=”\n”;

FS=”:”;

}

/^market/{

market=$2;

}

/^store/{

store_num=$2;

store_abbr=$3;

}

/^#/ {

print “”;

print “echo \”"$0″\”";

print “”;

}

/^run/{

dept=$2

start_minor=$3;

end_minor=$4;

print “btq_submit -c H prclst <

print store_abbr” ” dept;

print market;

print store_num;

print start_minor;

print end_minor

print “AAAA”;

print “ZZZZ”;

print “A”;

print “2″;

print “N”;

print “”;

print “END”;

print “”;

}

‘|sh

<– Start of the input section

<– end of input

<– setting up vars in AWK

<– Start of run command that will make the btq_submit block

How AWK Works

What AWK does is that it takes input and then brakes that input in to records and fields. It then preforms actions on those records and field and produces output. If you look at our input section each line is one record. Each field in the record is separated by a :

example

run:FURN:310:574Z\n

Is one record
- “\n” tells awk that this is the end of the record
Has 4 fields
- field 1 is “run”
- field 2 is “FURN”
- field 3 is “310″
- field 4 is “574Z”

Now if we want AWK to do something when it sees this record we need to make a block for it in the AWK command.

example

/^run/{

dept=$2;

start=$3;

end=$4;

print “dept is “dept”, start is “start”, end is “end;

}

The first part (/^run/) means: if the record start with run then do the following. The part inside the {} is the block of code to run. Here we are setting 3 vars from the feilds in the record. $[field number] is how you access the text in a field. $2 is the second field $3 is the third field … The last line is a print command. The sections in “” will be printed as text. Var names not in quotes will be substituted with there value.

The result returned by AWK for the above example is: dept is FURN, start is 310, end is 574Z

Parts of the Command

echo”[input]“|awk ‘[awk program]‘|sh

You must first echo your input and pipe it into the AWK command. You must then pipe the output of the AWK command to the shell if you want it to run the reports. You must have the “|sh” on the end if you want the script outputted by the AWK command to run. If you leave it off it will just output to the screen. This can be useful for debugging.

BEGIN block

At the start of the awk program there is a special block names BEGIN. The BEGIN block will execute before anything else in the awk program. In the begin block you have to set the separators that awk uses to tell the records and fields apart in the input. If you do not set these awk will default to RS=”\n” and FS=” “. Not normally what you want.

BEGIN{

RS=”\n”;

FS=”:”;

}

RS is the record separator

FS is the field separators

Setting Up the Vars

There are 2 ways to setup vars in this style of script. There is are persistent vars and there are run time vars.

Persistent

If you have one var that will be the same for several reports but not all reports you can use this method. If you look at the example the parts dealing with persistent vars are in green . In the input section they are written as [name]:[value1]:[value2] … you can have as many values after the name as you want.

In the AWK section of the script you can see the sections that set the persistent var. the way these are written is

/^[name]/ {

[name of awk var1]=$2 #field 2 from input line;

[name of awk var1]=$3 #field 3 from input line;

}

The part in green is how AWK knows what to do with the line from the input. AWK reads each line and then looks for any blocks to apply to that line. What the /^name/ means to AWK is “if the record starts with [name] then do the following”.

example
If line starts with “market” then set the var market = to the second value in that line.

/^market/{market=$2;}

Once you set a var in AWK it will retain that value until it is reset or the AWK program exits. So vars you set outside the run block may be used for several reports.

At run time

If there vars that will change with each report you can include then in the run line. If you do not want to place them in their own vars you can use the $[field] notation.

The run Line

This is where AWK will output the btq_submit block. This is done using the print command. There will be 1 run line for each report you want the script to run.

example

/^run/{
dept=$2
start_minor=$3;
end_minor=$4;

print “btq_submit -c H prclst <
print store_abbr” ” dept;
print market;
print store_num;
print start_minor;
print end_minor
print “AAAA”;
print “ZZZZ”;
print “A”;
print “2″;
print “N”;
print “N”;
print “N”;
print “N”;
print “”;
print “”;
print “END”;
print “”;
print “”;
}

If this line is ran it will produce an a result like the following.

btq_submit -c H prclst <
COT APPL
STL
31
250
309Z
AAAA
ZZZZ
A
2
N
N
N
N

END

Making Switch Vars

Some times you may have a report parameter that is only different once or twice in a large number of reports. It would be a lot of extra work to state this parameter for every report just so it can be different once or twice. In these cases you can use this technique to make a switch var. A switch var is different from a normal var in that it has a default value. If you set a switch var the value you set it for will be used by the next report ran. The switch will then be set back to its default value and following reports will use the default value.

echo “
date:1-JUL-09\n
run:123:123\n
run:234:234\n

S_sort:S\n
run:234:234\n
“|awk ‘
BEGIN{
RS=”\n”;
FS=”:”;

S_sort=”D”;
}

/^date/{date=$2;}
/^S_sort/{S_sort=$2;}

/^run/{
s_minor=$2;
e_minor=$3;

print “btq_submit -c A reportName <
print date” “s_minor”-”e_minor;
print date;
print “ALL”;
print s_minor;
print e_minor;
print S_sort;
print “END”;

S_sort=”D”
}

In the example above the lines for the switch var are in red.

As far as the input part of the template you use the switch var the same way as you would use a normal var.
- I like to name them starting with a capital S to show that it is a switch var and not a regular var.
In the BEGIN block you need to set the switch var to its default value.
Make a set block for the switch block in the same way as you would for a normal var.
it is used in the print btq_submit area the same as a normal var.
at the end of the run block set the switch var back to its default

You can have more then one switch var in a script, but you will need to have all of these parts for each one.

For loops

It is also possible to add looping to awk script. see the example below

#!/bin/sh

echo “test script \n\n\n”

echo “

date:3-jun-09\n

for_stores:MAD EST\n

run:xxy:123\n

run:yyy:234\n

for_stores:WAU OAK BDR;\n

run:yyx:345\n

run:yxx:456\n

“|awk ‘

BEGIN {

RS=”\n”;

FS=”:”;

}

/^date/{date=$2;}

/^for_stores/{split($2,stores,/ /);}

/^run/{

slp=$2;

minor=$3

print “######### start loop #########”;

for( i in stores){

print “btq_sumit report -c A <

print stores[i]” “date” and “slp;

print stores[i];

print date;

print slp;

print minor;

print “END”;

print “”;

}

‘

#!/bin/sh

echo “test script \n\n\n”

echo “

date:3-jun-09\n

for_stores:MAD EST\n

run:xxy:123\n

run:yyy:234\n

for_stores:WAU OAK BDR;\n

run:yyx:345\n

run:yxx:456\n

“|awk ‘

BEGIN {

RS=”\n”;

FS=”:”;

}

/^date/{date=$2;}

/^for_stores/{split($2,stores,/ /);}

/^run/{

slp=$2;

minor=$3

print “######### start loop #########”;

for( i in stores){

print “btq_sumit report -c A <

print stores[i] ” “date” and “slp;

print stores[i] ;

print date;

print slp;

print minor;

print “END”;

print “”;

}

‘

The output will look like this.

######### start loop #########

btq_sumit report -c A <

MAD 3-jun-09 and xxy

MAD

3-jun-09

xxy

123

END

btq_sumit report -c A <

EST 3-jun-09 and xxy

EST

3-jun-09

xxy

123

END

######### start loop #########

btq_sumit report -c A <

MAD 3-jun-09 and yyy

MAD

3-jun-09

yyy

234

END

btq_sumit report -c A <

EST 3-jun-09 and yyy

EST

3-jun-09

yyy

234

END

######### start loop #########

btq_sumit report -c A <

WAU 3-jun-09 and yyx

WAU

3-jun-09

yyx

345

END

btq_sumit report -c A <

OAK 3-jun-09 and yyx

OAK

3-jun-09

yyx

345

END

btq_sumit report -c A <

BDR; 3-jun-09 and yyx

BDR;

3-jun-09

yyx

345

END

######### start loop #########

btq_sumit report -c A <

WAU 3-jun-09 and yxx

WAU

3-jun-09

yxx

456

END

btq_sumit report -c A <

OAK 3-jun-09 and yxx

OAK

3-jun-09

yxx

456

END

btq_sumit report -c A <

BDR; 3-jun-09 and yxx

BDR;

3-jun-09

yxx

456

END

What is important to notice here is that each run line will submit a report for each of the vars that the for_ var is set for. It is also important to notice that the for_ var remands set after each run line. So multiple runs can be made with out reseting the for_ var.

100 Pushups

egoncasteel — Sat, 04 Jul 2009 00:00:20 +0000

I decided that after almost 5 years behind a desk I should really start getting some exercise again. Of course like most people I don’t like exercise, and after the a fore mentioned 5 years behind a desk I didn’t think I was ready to start big with some thing like PX90. So after after looking around I decided to go with the 100 pushup program. Its simple, doesn’t require equipment, and it doesn’t take a lot of time.

I just finished my 3rd week of the program which puts me half way through. I wont lie. I am sore a lot of days but I feel pretty good. I haven’t been able to do push ups like this since playing sports back in high school, and I am starting to look a little leaner and buffer in the the chest and arms.

I don’t know if I will make it to the 100, but the program hasn’t killed me yet. I will proudly post the ” I did the 100 badge” here if I make it. In the mean time I recommend the program to any one that wants to start small to get in shape.

Adventures in Home Computing

egoncasteel — Thu, 18 Jun 2009 21:32:57 +0000

2 months ago the hard drive in the little asus computer I had been using for watching video in my bedroom died. It put up a good fight lasting 2 or 3 years as my anime torrenting and watching / web surfing box. I had been planning on replacing it with my old gaming rig any way now that I have the laptop. As it didn’t have the power for HD video. I had just planned on doing it on my schedule not at the whim of corrupting hard drives.

Moving on to the point of this rambling. I quickly setup what I needed on top of the old windows install already on the computer at the time. I planned to reinstall again once some new parts came in, but was being lazy since things were working with only a bug here or there. Last night I found that my computer had gotten some nasty spyware on it. Not sure where it came from, but after threatening the computer several time that I would mind wipe it if didn’t cooperate with the spyware removal program. I decided to take vengeance upon it.

Every thing went just fine. I am no stranger to doing a full reinstall, and I was up and running again in around 3 hours. That includes reinstalling 20 or so programs and such.

The conclusion I came to after this is, it so much nicer doing this now that I only use open source programs and a couple of pieces of freeware. Back 8 or 10 years ago when I used office, photoshop, dreamweaver, … the process was such a pain. Find the stack of cds, get all the keys. Spend all this time with long long installs of bloated DRM filled software. Now I just download 100mb or 200mb of apps and go.

I don’t have any games on this computer but now that I buy all my games thou Steam even those install are a breeze.

New TinyMCE Version

egoncasteel — Wed, 10 Jun 2009 16:58:44 +0000

It had been some time since I had updated TinyMCE on OPSweb, but last month I finally got to it. I went from the last stable version of 2 to the latest stable release of 3, both with the appropriate versions of MCImageManager. Paste from Word is the killer feature that prompted this upgrade. That and the bug that popped up in version 2.

“I don’t know why it just started doing that. I was going to upgrade it any way let see if we get lucky and it just gos away.”

I am impressed. The paste from Word tool works fairly well. This is something that many people had asked about. You could paste from Word in version 2, but Words idea of a HTML export is a nightmare. If there is such a thing as anti-semantic HTML the crap the Word spits out would be it.

Oh and the bug went away.

Making something complicated simple

egoncasteel — Wed, 10 Jun 2009 16:22:00 +0000

I have made geometric solids before back in high school, but today I found a much simpler way to do it. “Slide-Together” Geometric Paper Constructions by George W. Hart shows how to make them by sliding card stock cutouts togeather. No folding.