HomePage | Optical Illusions | War Stories | QBasic | Dads Navy Days | Bristol | Bristol, USA | Bristol, Canada | Terre Haute | Miscellany | Web Stuff | About Ray | Site Map | Site Search | Messages | Credits | Links | Web Rings
QBasic | Errors | 40lb Weight | Bits | Chance | Colours | Dates | Delays | File Dialog | Files | Input | Matching | Menus | Mouse | Numbers | SeqNo | SIRDS | Sorts | Text | Timer | DLoads
SDF and CSV files
One of the most troublesome aspects about programming are data files and how to access them. At work, our preferred data format is SDF or space delimited, here the records are of fixed length, though each field within the record may have different lengths. Most of the utilities we use at work are geared towards this sort of file. Another common data file format we come across a lot is the CSV, (Comma Separated Value) comma delimited format. Here the fields are seperated by quotes and commas. Though some of the companies that supply data to us seem to think we can work wonders!
An example of an SDF file :-
SerialNo Name (18) Address Line 1 (25) Address Line 2 (25) Address Line 3 (25) 0000001 Mr A B Sample 123, Sample Road Sample Town Sample County 0000002 Mrs A N Other 456, Nother Street More Town Any County
An example of a CSV file (1) :-
"SerialNo","Name","Address Line 1","Address Line 2","Address Line 3" "0000001","Mr A B Sample","123, Sample Road","Sample Town","Sample County" "0000002","Mrs A N Other","456 Nother Street","More Town","Any County"
An example of a CSV file (2) :-
SerialNo,Name,Address Line 1,Address Line 2,Address Line 3 0000001,Mr A B Sample,"123, Sample Road",Sample Town,Sample County 0000002,Mrs A N Other,"456, Nother Street",More Town,Any County
An example of a CSV file (3) :-
SerialNo,Name,Address Line 1,Address Line 2,Address Line 3 0000001,Mr A B Sample,123, Sample Road,Sample Town,Sample County 0000002,Mrs A N Other,456, Nother Street,More Town,Any County
The CSV file (3) makes no query about there being commas in the data and so is treated as another field unless you're very careful. In the example above, if some records do not have the comma between the house number and the street name then extra fields are produced which plays havoc when processing these files.
All the CSV files have a CRLF at the end of each record. Some SDF data files have a CRLF at the end of the record and others don't. Some have the CRLF immediately after the text in the last field which makes it a SDF file in name only. Some data houses only put a CR and still others will only put a LF. The reason this may be happening is that PC text lines end in CRLF but Unix just inserts a LF and MACs just use a CR.
As you can see even amongst these two common data formats there is a lot of variation on what is supposed to be a common standard, and that's not even touching some of the other, more exotic types or EBSDC and other encoding with or without file headers.
Random, Sequential and Binary files
Files can be written or read as Random, Sequential or Binary files.
Random - the records may be accessed, edited and written to in any order. This is good for editing single records and rewriting them back to disk. Used mainly for data files.
Sequential - the records are accessed one at a time, they are also written this way, so if you want to edit record number 3,456 then all the records must be accessed until the correct record number is found, then the whole lot, including the records past number 3,456 rewritten to disk. At work, because most of our data files arrive on tape either 8" reels, DAT or cartridge they are read onto the data disks sequentialy, from there we can access them how we want. Used for data and text files.
Binary - any byte position within the file can be accessed, edited and written. Used for all disk files, but care must be taken as it is very easy to overwrite data with this method.
QBasic can deal with SDF and CSV files, and can read and write to files Randomly, Sequentialy or using byte positions.
Download sample data files as a zip file (4Kb)
The code that follows shows you a couple of ways of reading and writing to these files. Once individual records are read into the program any string manipulation that needs doing can be carried out using MID$, LEFT$, RIGHT$ etc. For the time being these programs will merely display the records.
File Prog 1 - Random file manipulation of WEB1.DAT
'A program to display the records WEB1.DAT, opened as a RANDOM file DIM Path AS STRING 'Holds YOUR path to the file being used DIM InCount AS LONG 'Counter for number of records input DIM FileNo AS INTEGER 'Gets the next free file stream DIM InRecord AS STRING 'Variable to keep each record DIM NoRecs AS LONG 'Gets the number of records in the file DIM FilePath AS STRING 'File path + file name Path$ = "E:\RAY\WEBSITE\DEVT\" 'Replace this with the path to where the 'you saved the data files FileNo = FREEFILE 'Determines the next free file stream RecLen = 114 'The record length of the records '************************************************************************** 'This piece of code gets each record in turn and displays the whole record OPEN Path$ + "WEB1.DAT" FOR RANDOM ACCESS READ LOCK READ WRITE AS FileNo LEN = RecLen FIELD FileNo, 114 AS InRecord$ NoRecs = LOF(FileNo) / RecLen FOR InCount = 1 TO NoRecs 'Using a counter like this actualy reads sequentialy GET FileNo, InCount 'Read each record in turn to InRecord$ PRINT InRecord$ 'Any data manipulation can be carried out here NEXT InCount CLOSE FileNo 'Closes the opened file stream PRINT PRINT "Press an key to continue . . ." DO LOOP UNTIL INKEY$ <> "" '************************************************************************** '========================================================================== 'This piece of code demonstrates we can get any record we want and using the 'FIELD instruction get just the bit of data we want. OPEN Path$ + "WEB1.DAT" FOR RANDOM ACCESS READ LOCK READ WRITE AS FileNo LEN = RecLen FIELD FileNo, 19 AS Waste1$, 30 AS InRecord$, 65 AS Waste2$ CLS PRINT GET FileNo, 2 'Get record number 2 PRINT InRecord$ GET FileNo, 5 'Get record number 5 PRINT InRecord$ GET FileNo, 7 'Get record number 7 PRINT InRecord$ CLOSE FileNo PRINT PRINT "Press an key to continue . . ." DO LOOP UNTIL INKEY$ <> "" '========================================================================== '++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 'The next piece of code demostrates how to read a RANDOM file using data types 'REMEMBER - If you change the record length of 1 record then you must change 'them all. If you don't then you no longer have a random data file but a load 'of junk!!!! 'The very useful FIELD statement is no longer available if you use Visual 'BASIC, the TYPE statement is used instead. QBasic offered both types of 'record input TYPE RecStruct SeqNo AS STRING * 4 Title AS STRING * 10 Initial AS STRING * 5 Surname AS STRING * 10 Add1 AS STRING * 20 Add2 AS STRING * 15 Add3 AS STRING * 20 Postcode AS STRING * 10 Number AS LONG Filler AS STRING * 4 Job AS STRING * 11 END TYPE DIM Record AS RecStruct CLS OPEN Path$ + "WEB1.DAT" FOR RANDOM ACCESS READ LOCK READ WRITE AS FileNo LEN = RecLen FOR InCount = 1 TO NoRecs 'Using a counter like this actualy reads sequentialy GET FileNo, InCount, Record PRINT Record.SeqNo; Record.Surname; Record.Add1 NEXT InCount CLOSE FileNo END
Sequential files can be manipulated the same way using a counter to loop through the records.
A common thing that needs to be done is to check whether a file is present before either trying to create a new one, or appending to a present one. I've written one method on the Errors page using ON ERR. Here's a method that was kindly sent to me by Pebe :-
SUB check4file(filename,a%) OPEN (filename) FOR APPEND AS #1 IF LOF(1)=0 THEN a%=0 ELSE a%=1 CLOSE filename IF a%=0 THEN KILL (filename) END SUB
QBasic | Errors | 40lb Weight | Bits | Chance | Colours | Dates | Delays | File Dialog | Files | Input | Matching | Menus | Mouse | Numbers | SeqNo | SIRDS | Sorts | Text | Timer | DLoads
HomePage | Optical Illusions | War Stories | QBasic | Dads Navy Days | Bristol | Bristol, USA | Bristol, Canada | Terre Haute | Miscellany | Web Stuff | About Ray | Site Map | Site Search | Messages | Credits | Links | Web Rings