• moonriver said on Jan 10, 2008....

  ed, i had exactly that problem in a project around 8 yrs ago. non-normalized data in 20 tables, with a total of (IIRC) 20,000+ records. the db system was in foxbase, and everything was kludgy. i was to migrate everything to ms access tables, with a VB front-end. it took me an entire month just to normalize the tables, clean up dupes and other record-level problems. that's where i found SQL very useful.
  
  i feel your pain, ed. i feel your pain... :-)
  
  

• silverwhisper said on Jan 10, 2008....

  omg...you know exactly my pain!
  
  a VB front-end? why not simply employ access--isn't that what most shops use it for, to make a front-end for SQL through ODBC?
  
  thank you, moon. your voice of experience is proving incredibly palliative. :>
  
  ed

• moonriver said on Jan 10, 2008....

  ed -- the nice thing about a VB front-end (i was using VB5 back then) is that it gave me very easy access to a lot of custom controls (OCX, VBX) and 3rd-party add-ons, which provided a wider choice of functions -- grids, trees, graphs, tabs within forms. plus crystal reports which i found very useful for the app we were developing at that time.
  
  for SQL, yes, it would be simpler to just use access and, from there, invoke SQL.
  
  

• tizzygirl said on Jan 10, 2008....

  This post went straight up and over my head....but yeah grrrrr.....what you said!

• MissMimi said on Jan 10, 2008....

  Ed!  Snap out of it, man!  You're sinking into geekdom!

  I'm with tizz on this one.

• silverwhisper said on Jan 10, 2008....

  TG: heh...thank you. :>
  
  mimi: OK, you do realize that i actually am a geek, right? :>
  
  ed

• MissMimi said on Jan 10, 2008....

  Maybe so, but at least before you hid it well.  This is just disturbing on so many levels.  ;)

• silverwhisper said on Jan 10, 2008....

  :D

• fearing said on Jan 10, 2008....

  That happens to me all the time Ed. 
  
  
  

• silverwhisper said on Jan 10, 2008....

  fearing: you're a database geek?
  
  ed

• fearing said on Jan 10, 2008....

  Being cute.
  ;-)
  

• TinSoldier said on Jan 10, 2008....

  I've been meaning to ask -- what exactly is "non-normalized data"?
  
  "Schema" and the bit about "60 tables that appear to duplicate discrete records within the master history table" is also confusing to me.

• silverwhisper said on Jan 10, 2008....

  fearing: i shoulda known... :>
  
  TS: non-normalized data...there's a reason database geeks say that "free text" is 2 four-letter words. let's leave it at that. :>
  
  ed

• TinSoldier said on Jan 10, 2008....

  So, fixed-length fields?
  
  Is there a "databases for dummies" book that I can read?
  

• fearing said on Jan 10, 2008....

  Isn't it what I do best?  Go ahead - you know that is your favorite thing about me right?  I'm just so darned cute!  Who else could make you cringe and smile at the same time so effectively?
  {cheesiest grin}
  

• silverwhisper said on Jan 10, 2008....

  TS: o, no...if only...let's just say that phone#s should appear in fields that are supposed to house phone#s. :>
  
  fearing: [trout-smack!] :D
  
  ed

• bloc said on Jan 10, 2008....

  normalization is a term for relational databases and it's core goal is to avoid duplication of data and maintain a good structure between tables. Think about it like this, if you have the same piece of info in more than one place it takes more space and it runs the risk of becoming inconsistent. 

• kelly said on Jan 10, 2008....

  Oh yeah, let's geek out!  I got a little confused because I thought you were talking about normalized data but instead I realized you meant tables that were in normal form.
  
  
  I can relate.  I don't have to deal with anything quite that bad, but I can say that one of the places I work actually discourages the use of multiple tables because they don't want to have to do joins and they're afraid they might end up with deletion anomalies.  Yikes.
  
  
  I wasn't aware that your job was dealing with a legacy database.  Sounds a bit painful, actually.

• crybabylu said on Jan 11, 2008....

  I am geeked out myself today, I had to reinsert a software program that did not want to go, and it is one I have used many...many times before,...but that was nothing compared to yours.......bet you will be glad to get that one worked out.

• fearing said on Jan 11, 2008....

  We were on the verge of honesty Ed and you had to go and trout smack me.
  Sigh.  Just remember, I'm the one who took the time to come to this post and try to relieve your stress from non-normalized data something-or-ruther.   Your blood pressure was up, you were getting testy and I alone distracted you!  I am victorious!
  
  What is non-normalized data anyway?  Is that the way to talk about data with socialization issues - NON-normalized?  It isn't normal?  .......
  
  Too much coffee for me this morning. 
  
  
  

• silverwhisper said on Jan 11, 2008....

  moon: sorry man, i completely neglected your last comment! OK, that does make sense to me. :> although i'd think access developers would be cheaper than VB developers...
  
  bloc: thank you, i was running low on time to explain properly. :>
  
  kelly: painful is precisely the word. :D wait--they discourage multiple tables? but...wow. aren't they experiencing issues with the database's footprint?
  
  dee: i will be very, very glad. :>
  
  fearing: in layman's terms, data stored in a database is stored in a structure called a table. in relational databases (most in use these days are of that type), you store certain types of data within different tables but tie them together, so that you can backtrack and relate the information properly.
  
  for example: let's say that you built a database for your contacts--family, friends, professional colleagues, etc. you might very well store that in one large table. many folks do. however, in a relational database, that's not desirable, b/c you have the person's name, their phone#s, their e-mail addresses, their physical addresses...all of those things might reasonably be put into their own respective tables: a table for phone#s, another for e-mail, a third for addresses (and potentially associated landlines)...
  
  now, in a properly-designed database, you would probably have a field (the structure within a table that actually holds data) for, say, phone#, another field that describes what type of phone# (home, work/daytime, mobile, work mobile, etc), and a third field that identifies to what person the phone# in question is associated.
  
  the problem of non-normalized data is that people will put, say, a phone# in an address field. this creates a problem b/c if you want to run a mail merge and send that person a letter, the resulting mailing address on the label will have a phone# in the middle of it. so you need to identify instances like that and somehow address them.
  
  the complication is that b/c i don't work with this database myself (in the sense that i don't do data entry into it), i don't know how often this sorta thing happens. if i expect [data] to be in a given field and it isn't there, i cannot reliably manipulate that data. it's especially problematic b/c if i can't be sure [data] is in that field, i have to hunt for what field it's in. and that isn't easy.
  
  so if it were just a contact database i was working on of my own--i could find those problems and fix 'em (although as something of a database geek i likely wouldn't have introduced those errors in the first place--but that's neither here nor there).
  
  but i can't do that here. and i have to normalize the data in order to import it (all of it) into the application i'm designing.
  
  TS: a very belated thanks for your patience.
  
  "schema" refers to the design/structure of the database: tables and the relationships between them, primarily.
  
  "60 tables that appear to duplicate discrete records within the master history table". there are 60 tables within the legacy app that collectively appear redundant with a separate (master history) table within the legacy app that (purportedly) tracks the history of adds/changes to the database. when i wrote that, the 60 tables appeared redundant. turns out they are, so i won't stress it. but analyzing 'em all took me most of yesterday.
  
  that data's important b/c the customer needs to know that history of change (i refer to that as HOC). it's pissing me off that it's all in multiple tables, though. i'm breaking out HOC by table in my app. while i may be over-designing here by doing that, their legacy app sure as hell was under-designed...
  
  this past tuesday i was visiting their site. the app's a dog! man, my 94 ear-old grandmother runs faster than this thing!
  
  OK, thanks for your patience, guys!
  
  ed

• TinSoldier said on Jan 11, 2008....

  Heh. You're so cool ed. Thanks for the info.

• silverwhisper said on Jan 11, 2008....

  bah, i'm not cool. i'm just mildly chilly. :> thanks for wading through all of that, i hope it answered your questions satisfactorily. :>
  
  ed

• fearing said on Jan 11, 2008....

  Ed you do a very good job of explaining geek stuff to blondes.  I actually got it!  ;-)
   

• moonriver said on Jan 11, 2008....

  pssst, ed, i know you know this trade secret... most of the basic functions could actually be done in access, but you know the psyche of some office bosses. they wanted a sexy customized program with bells and whistles, they were willing to pay more... and i already had lots of reusable code from previous vb projects... *wink*
  
  pssst, fearing, non-normalized data are data that suffered emotional deprivation when they were babies ... :-)
  
  

• fearing said on Jan 11, 2008....

  @Moon, I thought there were emotional issues at the root of this.  Sad man, just sad.

• silverwhisper said on Jan 11, 2008....

  fearing: i was worried it was too much--thank you for reading that! :>
  
  moon: i must study at your feet, sir. heh...emotionally-deprived data... :D times like this i really wish SC had slashdot-esque karma points... :>
  
  ed

• moonriver said on Jan 11, 2008....

  fearing -- exactly. and ed is really a psychologist in computer geek garb. he's putting all those emotionally-damaged data tables into a 2-week regimen to make them normal again... :-)
  
  ed -- i need some of those karma pts right now... :-)
  
  
  

• silverwhisper said on Jan 11, 2008....

  moon: o, i don't know about 2 weeks...these are some deeply, deeply non-normalized data we're talking about here...it may require more aggressive measures. :D
  
  ed

• fearing said on Jan 11, 2008....

  Ed, it wasn't too much at all.  I appreciate you taking the time to explain it. 
  
  Moon, It is nice to see Ed using his talent for the greater good.  I think he could have his own show - Dr. Ed.  ;-)
  

• silverwhisper said on Jan 11, 2008....

  o, i couldn't be a doctor, fearing, that would perhaps be too traumatic and threatening for non-normalized data. i would just have to be a mister. besides, i've already had my own show... :D
  
  ed

Comment on "rant: i hate hate hate non-normalized data"

Currently tagged	databases rants database schema database architecture work (Click to add tags below)
Tag this post	(Separate tags using commas, for example: New York, dating, vegetarian)
Your comment