I agree, even if Kalmbach is unwilling to part with the index, nothing stops the same interested volunteers from starting their own open source index.

 I'm sure now, we have had so much interest in this, whether Kalmbach decide to release the original data or not, we will be able to  rebuild the index, hopefully better than it was before.

I've  asked elsewhere for advice about  what software we could use for our new index. see - http://ask.slashdot.org/story/10/07/17/1928234/Developing-a-Niche-Online-Content-Indexing-System

Although I've still not found anything off the shelf that would be ideal for this I have found several interesting possibilities. There is a wonderful range open source programs already available. 

What we want to do  is very similar to a library indexing system but most of these seem to be part of intergreated systems.

There is a pre-existing standard for cataloging  bibliographic references - Bibtex. Using this, though it's not an exact match for our wants(it's designed for academic works),  would help ensure future compatibility with other systems and might ease the next conversion in another 20 years time!

A bibliography indexer might be a good choice. A free one is  Aigaion ,but this is  targeted at databases of scientific literature, so might need some tweaking. I've downloaded this and set up a test system at  http://pc-cafe.co.uk/mr/index.php/ . User and PW are both test if anyone wants to look at this..

There was this very inspiring talk by a man who wants to digitise every book ever made http://www.ted.com/talks/lang/eng/brewster_kahle_builds_a_free_digital_library.html

His Open Library's system  might be a good base to use, but having read the installation instructions http://openlibrary.org/dev/docs/setup I can't work out how to do this on a shared server and I'm a computer person !

If we do have to recreate the index from scratch it might be easier to scan and OCR all the magazines. There are some very good solutions for indexing  large bodies of text.

I have the LIbrary Master's degree, my first suggestion concerning library software is to simply ignore it!

I'd suggest building it up using a relational database, and then use an Apache server to interface the raw data with SQL search queries via a PHP based website.

If this was done right, I'd be able to look up articles by very specific details, or if i was jsut browsing, I could also look at results that are very broad  - like a page that displays contents by journal title and year/volume/issue - but well sorted!

I'll leave this to the codeheads to think through - But I did come close to designing an ERD once upon a time.  Maybe I can ressurect it...

t's really easy to create these things for the right people wh odo it day in day out like my professor - he could crack something out in about an hour, a blank text file to a fully operational database displayed on a website...I've sene it done by the pros, so I know the pros probably have a god handle on it once they decide to go ahead!

Thank you Joe, et al, for making this transition happen (if indeed it goes through) and thanks in advance for all the good, hard work that will be needed to rebuild or whatever must happen to keep the index alive.  I have been a longtime user of the index since before Kalmbach purhchased it and find it invaluable.  As a younger modeler who appreciates the 'old ways' and pioneers of the hobby, it's not just useful, it's impossible to calculate how much time it saves (vs. sifting through old issues one by one or creating my own index at home).

Regrettably I don't have the computer know-how or technical chops to contribute, but if there's anything I can do please tell me.  I'll be glad to help however I can.

I too am disappointed that the folks at Kalmbach didn't choose a solution to the problem of keeping the index going, and instead chose to shut it down.  I don't claim to understand the ins and outs of the publishing industry.  But as a modeler, and thanks to forums such as this, I know we can all voice our opinions and offer suggestions and as such have every right to criticize (constructively, of course) and even vent a bit.

Thanks again, Joe, for taking the lead on this.  Thanks to the NMRA as well for willingness to step up to the plate.  Now come on, Kalmbach, do the right thing for the modeling community.

Galen

I really like the Idea of a Wiki Railroad & Model Railroad Magazine Index!

With the Wiki system we could include every Index for every Model Railroad Magazine and Full scale railroad publication that has ever existed or will come to exist.

With the wiki format any one can register for free and add to the content of the index. Of corse there are moderators that double check the applied information for accorcy and also delete any Copy righted imformation other than actual indexing as to where said imformation is located such as Publication: Year, Month, & Day of puplication as well as page numbers where imformation needed can be found and a cross refrence to any other Imformation on the same subject mater may appear in other publications.

Now this can't just happen over night but once the main Index is in Wiki the cross refrencing can be added as it is found or needed by any regestered user of the Wiki Railroad Magazine Index.

Also one need not be regestered to use the index but only to add content to it as is in all Wikipieda subject Dictionarys.

Dan

 Let me explain why I'm looking at pre-existing software rather than just knocking up a quick custom suite of our own. Yes the basic structure would be fairly simple, we are probably talking about three database tables and half a dozen PHP/HTML pages to view the data in various ways, plus a few more pages to input or update things.

Now that's probably not more than a couple of days work for a half competent programmer(proceeded by 2 weeks of committee meetings to decide what they should contain) but, wait a minute, we where going to have this as a wiki so lots of people can enter the data, so we need to have log-ons and differing levels of access , Oh and an audit trail... and don't forget we need to make sure the Author's name is entered in a consistent way..........

You can see where this is all going. Suddenly it's not such a simple project any more.

Another thing using a package might give us is use of standards. This may not be important now but in another 20 years we don't want to be back in the same position again. If we have used industry standards there every chance the will be a program, written in what ever language is in vogue at the time, that we can import our data into.

A lot of this is just me thinking aloud, we are not going to be able to make any serious decisions until we know whether Kalmbach are going to give us access to the original data .

Another suggestion that's been made, which I am starting to like a lot is this -

“ Couldn't you just scan and OCR the magazines and then use that data to compile a searchable database? You could even supply extracts from the articles where the search terms appear, similar to what Google does. By not presenting the full text of the article I think you would be on safe ground copyright wise. “

I think that given the advances that have been made in search technology this may be a much better method than manually re-entering keywords. For those magazines that the publisher would allow we could even let people download a PDF of the article, I mean, this is no more than the NMRA library does now, abet with paper copies.

I programmed Internet protocol stacks for DOS PCs in the late '80s and early '90s.  My current job involves both DB design using SQL and picking other people's file formats apart.  Can anyone get the details of the environment the existing software uses?

James

Hello everyone, I would have to agree with the consensus here and say lets rebuild... bigger, better, faster...

As a professional web designer and model train enthusiast I would be willing to put my money where my mouth is and say I can build the site needed for this new database. I use predominantly open source software for building my clients sites and I believe I have just the thing for building this one.

It is a flexible but fixed format database for storing information such as this index. it include comprehensive and expandabe search functions. I also have the hosting servers  necessary to host and support the system once built. (hopefully with a few donations to help cover costs, but this is not important to me)

If we can get a consensus on the information we want stored and a presentation format I could get started on building this almost immediately.... Once complete anyone would be able to register and submit information, but only moderators will be able to make that information public, so it will be a joint venture amongst a group of us to maintain and expand the database.

Let me know what you all think and lets make this happen....

Wayne

tebee, those issues you bring up are challenges that would have to be crossed no matter which bridge we cross - and very valuable thoughts towards developing the final model!

The issues of access, accounts, logins and so forth is not as difficult as put forth.  This site has those very elements to it; in the module that I studied back in database class, we looked at models where the user could access the data and then models where the user could edit the data.  The differnece between the two is that the editor could access the input form, form which data could be entered.

Your last thought about data consistancy is very important.  Whatever database formate we use, it must be adequate to address this issue of name format and be able to accomodate all various formats of names without losing original data.  The last project I was on did not do much data intergrity enforcement.  There are ways to do this and I put them forth now.

One table in the database would regulate the progress of each record in the system.  Every time a reference is added to the database, the database would flag it as a "new" record.  Once a second [third, and/or even forth?] set of unique eyes looks at the Master record through the edit interface [editing a master record essentially loads all of the data information into the Add/edit input form]  The table flags the data as "read."  Once a site moderator has viewed the data once and pressed the submit button once more, the data at that point gains the "OK" flag which notifies future administrators and future editors which files have been viewed by peer-editors and gone through the peer review process and should need no further check.

We did one more thing back at RRUFF:  we added first one timestamp to each entrya nd then a second!  the first time stamp is a "created by."  The second timestamp is a "last edited by."  These entries basically record who did the action and when they did it.

The reason we would want to design our own database from the groundup is because no matter what package we use, it will have inherent weaknesses due to the intended use versus our use - unless there is specific package already like this in exsistance.  It would give use the most control from a data presentation persepctive as well as a data entry point of view.

Hi,

yesterday I did some research, basically inspired on the comments on Slashdot. It looks like we're looking for a solution that is similar to an Online Public Access Catalogue (OPAC) used by libraries.

Several impressive solutions are available that even are open source. One that caught my eye is VU Find. First hurdle to overcome would be using the domain specific file format they are using in libraries. This is called MARC.

Any alternative would be to start with the search engine VU Find uses. This is called SOLR and has some impressive features.

All this would work for standard searches. But there is one domain specific solution in the old index called trackplan search. This search handled scale conversion automatically and allowed a range for the size of the plan.

So if you were looking for an HO scale plan in 4'x8' an O scale plan that was 8'x16' would also be found. This is a very specific search function that we'd have to add ourselves.

Don't forget that this only covers the search part of the application. Everything that is related to data entry will need additional work. I think that using a community approach similar to wikis would be the only viable solution. But then we need to have a frontend that makes sure that data entry is as consistant as possible.

I think this would be a very challenging and interesting project. I wonder whether we're able to pull together a team that would start working on design of the overall solution.

Regards
Martin