![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png){kind=small}
Unexpected fuzzies
Dec. 1st, 2010 06:30 pmSpent a large chunk of today on the phone to one of our SAS support guys discussing a problem I've been having with one of my programs*. Ended up sending him the program + logs so he could look through it for himself, and he actually called it a "work of art". Go me!
He might have been thinking Picasso, or possibly Escher's "Ascending And Descending" given how long the wretched thing takes to run, but I shall take it as a compliment.
I know I'm not a real programmer, but in the course of developing some SAS training courses I've learned a little bit along the way... also, developed a sense of "if I am telling newbies how to write programs I probably ought to take some of my own advice" type guilt, which is a powerful motivator to plan, document, and validate.
The stupid thing is that I'm pretty sure this program was working two weeks ago; the thing that's been giving me all this grief is the validation step that confirms that it works in practice as well as in theory. Apparently non-optimisable SQL joins are a bad thing when the tables involved each have about a million entries... but I have wrestled it down to the point where it only takes about 10 hours to run and about a gig of memory!**
*I am about 95% sure the cause of the problem is "some other process, probably their automated backups, decided to put a lock on one of my files at the same time my program needed to write to it", but they wanted to explore other options.
**Which would be why I end up running it at night, at the same time the automated backups happen...
He might have been thinking Picasso, or possibly Escher's "Ascending And Descending" given how long the wretched thing takes to run, but I shall take it as a compliment.
I know I'm not a real programmer, but in the course of developing some SAS training courses I've learned a little bit along the way... also, developed a sense of "if I am telling newbies how to write programs I probably ought to take some of my own advice" type guilt, which is a powerful motivator to plan, document, and validate.
The stupid thing is that I'm pretty sure this program was working two weeks ago; the thing that's been giving me all this grief is the validation step that confirms that it works in practice as well as in theory. Apparently non-optimisable SQL joins are a bad thing when the tables involved each have about a million entries... but I have wrestled it down to the point where it only takes about 10 hours to run and about a gig of memory!**
*I am about 95% sure the cause of the problem is "some other process, probably their automated backups, decided to put a lock on one of my files at the same time my program needed to write to it", but they wanted to explore other options.
**Which would be why I end up running it at night, at the same time the automated backups happen...
