[sebhc] H8D files and labels

[Dave Dunfield]

This is really getting out of hand - I started this in response to someone
discussing how they would keep the labels for the simulator images.
I proposed a VERY SIMPLE modification to the format that I would be
willing to make to my simulator which would allow it to accept an optional
comment .... But all I have gotten in return is how we really should change
everything completely.

Ok --- I GET IT --- Nobody likes the idea.


> I'm moderately against creating a new image storage format.  It seems
> that each vintage group eventually starts to re-invent image formats.
> And each crosses over the other inventing formats that are "exactly
> the same but different."

You can't stop people from doing their own thing - especially if the
alternatives are completely at odds with your way of thinking (like
storing binary sector data in octal-ASCII). So I don't have a problem
with this.

What I do have a problem with is people developing their own
formats and NOT documenting them. Anyone who's ever had a
TeleDisk image that refused to turn itself back into a physical
diskette knows what I am talking about... If I have decent
documentation on how the image is formatted, I can explore
whatever means I find necessary to recreate a disk ...


> In my work with the SVD, I find myself in the position to deal with
> many different formats and creating software that works with each.  As
> such, I've developed rather rabid opinions about floppy image formats,
> with the basic guiding principal that the formats need to be useful
> for generations to come.  I mean, if we're going to the trouble of
> "saving" the software/info that resides on our dying floppy disks, we
> should save it in a format that anticipates future needs.

You can't anticipate every future need - so you develop a format which
works for what you can anticipate - often this is biased by the intended
use of the image.

In my work with ImageDisk, CPT, NST, RT and all the other disk transfer
utilities I have developed, I have formed a number of different formats
which best (or at least adaquately) serve the purpose for which they
were intended.

Take ImageDisk for example - it is designed to replace TeleDisk (with
a documented format), and attempts to replicate EXACTLY the soft-
sector disk that was originally recorded - to this end, It records the
exact sector numbering and ordering, normal/deleted address marks,
maps for cylinder and head numbering to accomodate non-standard
formats, and quite a bit more - in short, I record everything I can
determine about the disk from the PC's FDC in the image file. I also
have a clear file identified with the IMD version and creation timestamp,
and an unlimited size comment field.

This is probably pretty close to what you describe...

But I did not implement H8D with these goals.

I don't know who first make these images, however I implemented
the raw binary image for my H8 simulator, because I wanted to
experience running HDOS - I have still not actually seen a physical
H17 controller. The images I found in the SEBHC archive were
in the format we now call H8D, and they were relatively easy to
get working, so I used them "as is".


> My credo for archival of floppy formats is pretty simple (and in
> priority order):
> 
> Completeness - a floppy storage format for a particular target
>  system/OS should completely describe all of the data required
>  for that target.  For example, a good floppy format for H8/9
>  should include the volume number of the disk.  As another
>  example, often in the TRS-80 world copy-protected disks
>  included weird sector/track numbers; a good floppy format
>  should include that information.
> 
>  As a counter example, the H8D format is simply a dump of the
>  sectors of a floppy.  It is assumed that they are dumped in
>  order, and that the sector/track/volume numbers for the image
>  are easily discernable.  Although this works mostly, it forces
>  us to look into the data on the image to discover important
>  information like the volume number.  The H8D format is bad for
>  CP/M images because the volume number can't be easily found.

As noted above, the .H8D images I obtained were already in this
format, and having experience with NorthStar and not H17, I did
not know about the header bytes in the sectors until I was so far
into it that I didn't bother to change it.

It I were implementing it from scratch, I would probably have added
the sector header information, but I would still have written the image
in linear ordering - thats because I implemented the disk image for
use with a simulator, and this makes it very easy to "seek" to a
particular track and sector.

I do not see H8D as being the means to archive "exact" copies of
a disk - but it works pretty well for a simulator, and DOES preserve
the content of the disk very well. I see no reason to physically
interleave a simulator disk image, as interleave is a physical
ordering which will not affect the logical layout of the disk (which
is all the simulator sees).


>  If you take a look at the "other" H8/9 format with the
>  extension of .h17 (hereby called the H17 format) you'll see a
>  more "complete" format.  It describes the volume number as
>  well as sector and track numbers.  It includes, as well, a
>  "Description" field which is equivalent to the comment field
>  people have been asking for.
 
>  Granted, you can go overboard.  There is really no reason to
>  capture the individual flux transitions of a floppy image,
>  for example.

There is if you want to reproduce "exact" copies of the disk -
thats my whole point... You have to pick a boundary between
preserving "nothiing" and preserving "everything" - the exact
position of that boundary will vary depending on your intended
use for the image format.


> Self-Descriptive - an issue that I have smashed into over and over in
>  in working with the SVD, is that many file formats cannot be
>  identified uniquely regarding the system they target.  That
>  is, I can't necessarily tell the difference between an H8D and
>  the TRS-80 format JV1.  Fortunately, the files themselves are
>  often named to identify them, like the file "HDOS.h17".
>  Though you can't count on that.  Many groups like to name
>  their floppy images "blah.dsk" - which makes it often
>  impossible to figure out the target system for a floppy image.

The H8D header (like the IMD header) that I proposed for the
(slightly) expanded format would correct this.


>  This issue is closely linked to the "completeness" issue
>  above.  In almost all cases, you can't be fully "complete" and
>  you need some target information in order to "fill in the
>  gaps" when using an image.  For example, even with the H17
>  format the file doesn't describe that the original media was
>  hard-sectored.  So if you wanted to faithfully reproduce the
>  "HDOS.h17" disk, you have to KNOW that it was on hard-sectored
>  media using the H8/9 sector format.

This isn't a problem for me - the simulator for which I implemented
H8D does not deal with soft-sectored.

This is my way of saying "this is a SIMPLE format I implemented
to create a virtual H17 in my simulator" - Yes, you can rename a
.XLS file to .H8D and "stuff it in the drive", just like you can put a
NorthStar disk into your H8 - and you should accept similar
results - the real H8 wouldn't say "hey - thats on Osborne disk" -
It simply wouldn't boot.

However, having said that - the unlimited size comment that I
proposed to be added to the H8D format would make it possible
to describe the content of the disk image and the media type
in excrutiating detail - you can even mention the color if you
like.


> Size - many people get rather religious about the size of image file,
>         and that difference can be rather big.  For example, an image
>         in H8D may be 100k bytes while the same H17 image may be over
>         400k.	Although this is indeed an issue, it is an issue that
>  continues to get less important as storage becomes cheaper and
>  cheaper and speeds faster and faster.  I will always prefer an
>  image format that is more complete or descriptive than one
>  that is smaller.
> 
> The above list was in priority order for me.  So I could really care
> less about size...or maybe more accurately, it can be dealt with in
> other ways than to trade off completeness for size (for example).

Size matters if you are on dial-up (I am).
Size matters when you want to put a large collection of images
on CD.
Size/speed matters when the simulator has to encode/decode
sectors in real-time when it could have just loaded/stored the
binary image.

I'm not against justified overhead, and I agree that the H8D format
could use the few extra sector-header bytes to make it more
flexible, however I simply cannot see why 3-4 times the entire
image file size would be justified.

 
> So to bring this back to the topic at hand.  Personally, I'd be for
> reverting the SEBHC collection back to H17 format as opposed to
> expanding the H8D format.  Or maybe a common format could be used.
> For size concerns, while they still exist at this level, "zipping" 
> of the files would be easy.


Well ... I have no interested in implementing .H17 - Encoding sectors
in ASCII is "just plain wrong" to my way of thinking. But I'm not overly
attached to the current H8D either - all I contributed was the extension
name.

You can encode images in whatever format you wish, and in whatever
code you wish - you can use EBCDIC, BAUDOT or Selectic tilt/rotate
codes for all I care - as long as you document it, I can extract and convert
it if necessary - thats all I really care about.

Having the ability to marry the description to the disk image would have
been convienent - but enough already - it's clear that this is not what the
group wants, so I will withdraw the suggestion.

regards,
Dave

--
dave06a (at)    Dave Dunfield
dunfield (dot)  Firmware development services & tools: www.dunfield.com
com             Collector of vintage computing equipment:
                http://www.parse.com/~ddunfield/museum/index.html

--
Delivered by the SEBHC Mailing List