Jump to content
 

D1001

Members
 View Profile  See their activity

  • Posts

    53

  • Joined

  • Last visited

 Content Type 

Blog Entries posted by D1001

  1. D1001

    BRDatabase availability

    By D1001,


    Keeping me informed
    Once again, my website is unavailable, but this time it is thanks to facebook. I don't fully understand this but facebook is indexing my website over and over again (it's a known thing) and has consumed all of my bandwidth for March (and it's only the 13th!) The hosting company wants me to go to a dedicated server which is like using a sledgehammer to crack a walnut. The cost of running it would triple from $18 to $48 per month (and that's just the starter offer). I can't afford that and I can't expect donations to cover that either. So, I have added some code to try and kick facebook straight off my pages but I still have consumed all my bandwidth so I cannot see brdatabase being available again until April.
     
    I am re-writing the website (it'll make it mobile friendly) and I am hoping to deploy to AWS where I may have more control over things. In the meantime, I am sorry for the incovenience.
     
    If you have any urgent queries, drop me a line and I'll look on my local copy for the relevant data.
  2. D1001

    BRDatabase Problems

    By D1001,

    The BRDatabase website has had connection issues recently and this is a quick headsup.
     
    The hosting company, who have been less than helpful, have denied access to parts of the database because of high cpu and high RAM hits. I fixed some errors recently and this has meant a refresh of the cache files which is putting a heavy load on the server. I am in the process of removing some of the worst queries and replacing them with 'data warehouse' tables which essentially means that the server will be under a smaller load.
     
    I am looking to move away from the hosting company - they recently increased their basic plan cost by 500% and removed interactive chat support. So I am looking elsewhere, but the cost of running the website has increased considerably since I started. If anyone has any recommendations, please let me know.
     
    As for the data, I have completely refreshed everything (offline) from 1946 to 1968 with the exception of a handful of months in 1965/6 which I will add in the next week or so. Then I will check the quality of the data and once that is done, BRDatabase should be much more accurate and reliable. Also, the facelift is imminent ...
     
    In the meantime, I apologise for the problems everyone is facing and I hope to get them resolved as soon as possible. That might mean a bit of downtime for maintenance, so please bear with me.
     
    Ian
  3. D1001

    Snapshot extended to include boiler and tender type

    By D1001,


    Tender snapshots
    I have noticed several discussions where members have been trying to determine what tender type was attached to members of a class at a particular time so that it would be possible to get an accurate snapshot of the configuration of a loco for a realistic depiction on a model layout. I looked at the existing snapshot page on BRDatabase and determined that it would be fairly straightforward to add this functionality where this information was already on the database. So, for instance, the tender information for the GWR Castle is already loaded and so a snapshot of tender type is possible for that class. This also applies to the LNER A3 and A4 but not much else at the moment.
     
    To see what I mean, run a query on this page: https://www.brdatabase.info/locoqry.php?action=class&id=146003&type=S&page=snapshot and the current tender type is displayed. I have also extended this to show boilers where known.
     
    There are some minor bugs (like tender types displayed for long scrapped locos) and I will address those shortly.
     
    Although this feature is very limited at the moment, it should improve now I know it might be of use.
     
    Please let me know what you think!
  4. D1001

    Loco Modifications

    By D1001,


    Keeping me informed
    I've recently been adding lots of info regarding the modifications made to diesel locomotives, such as dual braking, boiler status, livery etc... If you remember the spotting books back in the day used to publish a code indicating the current status of a loco, such as xo (dual braked, no heating) or vb (vacuum braked, working boiler). This is what I am aiming for. So, it is currently a work in practise (practice?) but will hopefully give modellers a clue as to what was what and when. Class snapshots have been enhanced so you can see the config of an engine at a particular date. As I said, though, it's a WIP.
  5. D1001

    BRDatabase data accuracy

    By D1001,


    Keeping me informed
    Hi,
     
    I thought I would just put a blog entry for people to highlight any problems they find with www.BRDatabase.info
     
    If you see an error, an omission or have ideas for how the data could better be presented, please leave a message below and I will see what I can do. I really want the resource to be as accurate and understandable as possible and I cannot do that without help - I regularly see messages saying "don't trust BRDatabase" etc... but I cannot fix everything by myself, so I would appreciate a more interactive approach to fixing problems.
     
    Thanks
  6. D1001
    One thing I need to put to bed once and for all: on my site, www.brdatabase.info, two GWR 0-6-0PT's, nos 9700 & 9701 are listed as having been allocated to Stratford (ER/LNER) in the 1940's. This is a parsing error. Both of these engines, at the time of allocation, had an GER/LNER equivalent, and it is these engines that should be listed as allocated to Stratford. It's the kind of mistake anyone can make after having consumed an eclectic array of real ales!
     
    As others have surmised, 9700 and 9701 refer to N7's. 
     
    A Collett Goods 0-6-0 No 2238 is also listed in 1946 - this should have been Startford-Upon-Avon.
     
    Incidentally, a Hawksworth 0-6-0PT No 9401 *was* loaned to Stratford in 10/1/1957 - 2/6/1957.
     
    The fixes for the three mistakes are not as simple as you would expect so I will fix this perhaps over Easter.
     
    Ian
  7. D1001

    BRDatabase status

    By D1001,

    Hi all,
     
    I have had a few enquiries about the state of BRDatabase recently. I haven't updated the site for several years and there are known problems with it. My enthusiasm for the project has been low in recent times - this has been primarily due to my youngest daughter being seriously ill throughout 2015, and I have been unwell myself this year.
     
    I mentioned in an earlier post that I had done lots of work behind the scenes, including better quality data - since then I have spent a lot of time at the National Archives in Kew, obtaining accurate information from their records. The data is comprehensive but also there is tons of it. Therefore, putting it into the website is a massive task for one person.
     
    As far as I am concerned, the website is just in the doldrums and sooner or later I will get the enthusiasm back to add more data and iron out the various bugs (like Scottish Granges!). In the meantime, I am happy to field any questions that may come my way and I thank you for your continued support.
     
    Ian
  8. D1001

    BRDatabase status

    By D1001,

    Hi, just wanted to give a heads up on BRDatabase.
     
    At the moment, part of the site is disabled (the sites data) because of the heavy workload inflicted on the servers (my inefficient code to blame). I have two options:
    Re-write the code to be more efficient (using pre-prepared data, in the style of a data warehouse)
    Leave it alone pending new website

    I am in the process of improving the data immeasurably, by redoing the data using SLS, Railway Observer, Trans Illustrated and other sources (e.g. Yeadon and Peto etc...). I am also looking at a complete revamp of the website, possibly including a member section (probably free). which will allow customisation, page history and favourites.
     
    There will be far more information pertaining to individual locomotives, including sightings (shed visits etc...), parsed from old railway magazines.
     
    Also, I will at long last fill the gap between late 1949 and the end of 1950 which has caused so much confusion over the past few years. I will also get the data for diesels, electrics and units in place right upto 1997 (and eventually into the 21st century).
     
    Because of the amount of work I have to do, I will get the sites data up and running again in the old format, pending new development.
  9. D1001

    An introduction

    By D1001,

    Hi,
     
    I am the author and webmaster of the website BRDatabase.info and as it gets mentioned fairly frequently on RMWeb, I thought I would try and give you all a better understanding of what it contains, how it has been built, the problems you may encounter and the plans for the future. I'm not going to do all that in this first post but I will try and keep you up to date with developments.
     
    Firstly, despite there being no major updates since spring, I am working on it all the time on my own local copy - I have been trying to improve the granularity of the data (i.e. have exact withdrawal dates where known rather than a month and year). This is very time consuming but will be worth it (I hate inaccuracies!). I am also trying to improve the performance as currently, any query is done on a RDBMS, and as the data is essentially static, I am identifying all of the potential data warehouse scenarios.
     
    Secondly, I get a little bit of flak from people saying that the site is inaccurate and should be treated with caution. To an extent, I agree with these comments but I must emphasise that it is a work in progress and that there are substantial holes which I am working hard to fill. For instance:

    Allocations between late 1949 and late 1950 are not yet fully loaded - this is causing depot snapshot and allocation histories to be inaccurate as it might appear that a locomotive is based at a pre-1949 allocation whereas in fact it moved during this 'black hole' period. This is my top priority to fix.
    Diesel and Electric allocations post 1976 are very lightweight and sketchy - I have an early Harris book but it is a nightmare to electronically parse. I also have the first 100 or so Rail Enthusiast magazines (as it was) and I need to plough through these at some time soon.

    Most of the initial data was very basic and was then augmented with scans of Railways Illustrated 'Motive Power Changes' - if there was an error in there, it appeared on the website. As time has gone by, I have found numerous sources of info, some of which conflict with others and I have to convert the data to the internal database format and, if necessary, make a call on which source to use.
     
    The sheer volume and intricacy of data means that I, as an individual, just simply cannot verify everything - that has to come from you. Rather than make critical comments about the quality of the data, why not send me a message and let me know what is wrong and where I can find the correct data.
     
    I'll leave it at that for the time being - I have plans for the future that I will mention soon, I also have requests for information - I want BRDatabase.info to be an accurate resource that people find useful, be they modellers or otherwise.
     
    Thank you for all the support I have received and the donations made to help me run the site and research the data.
     
    Ian
  10. D1001
    I have spent some time entering locomotives into BRDatabase.info and I thought I would share with you the reasoning and procedure, so you can see what goes on in the background.
     
    I was entering data for 1949 allocations and withdrawals etc from a copy of the Stephenson Locomotive Society journal. I came across an engine that didn't appear in the database - the reason being that I had it's proposed BR number but not it's LMS (ex L&Y) one, and it was withdrawn still carrying the LMS one. So, before I went any further I decided to look it up in my Baxter 3B - it turned out to be an ex L&Y Class 25 0-6-0, ordered by Aspinall to a design by Barton Wright. I had 126 of these locos in the database (including those that were converted to saddle tanks - 0-6-0ST, L&Y Class 23) but 280 were built, 240 of which ended up as tanks.
     
    My rule regarding which engines to include and which to exclude from the database is simple: if any loco from a class was in service on 1st January 1923, include the *whole* class. Therefore all 280 locos had to appear!
     
    So, I had to insert 154 new locos and update the remaining 126. I like to have contiguous serial numbers for a class of loco and also, gradually, I am renumbering each class (internally) so that the parent company and wheel arrangement can be derived - the Class 25 had an internal serial number 401 which I have changed to 406003 (4 for ex LMS, 060 for wheel arrangement and 03 for unique occurence of ex-LMS 0-6-0's), and class 23 was 399, changed to 406052 (the 5 indicating a tank loco). Complicated? It gets worse.
     
    Then, I scanned in all of the data from Baxter for the 9 different lots of Class 25 0-6-0's from 4 or 5 different manufacturers. Parse errors are normal and alot of correcting, formatting and aligning (using a text editor and excel) is necessary. I then added a unique locomotive id (internal for the database) - the new format is 9 digits, the first corresponding to the class id as built and the remaining 3 being unique to the loco, so the first built (L&Y 528) became 406003001 and the last engine (L&Y 977) became 406003280. However, 126 engines already had serial numbers of the old format (6 digits, sequenced in the order in which I originally built the database). So, I loaded the spreadsheet into a temporary database table using phpmyadmin, added an old loco id field and populated that by matching the Baxter British Railways numbers with those I had already on the database. I have a database 'stored routine' to renumber the id of a loco so I had to run that 126 times against the locos already present to give them their new id numbers. This involves changing the id in the main table and the rebuilds/allocations/mods/numbers/sightings/works_visits etc... tables too).
     
    Finally, I was able to insert and update data based on the temporary table of Baxter data. This includes entries in a rebuild table which changes the class id. This is for the 240 saddle tank rebuilds. Once all this is done, I run an update script (most of the tables are base tables and the update script builds tables that are eventually presented on the website - this was done in order to avoid repetition - for instance the main table and number table have dates in them - if the build date and the first number date were different, things went wrong, so now the number table uses the date from the main table for the first_number flag being set to Yes.)
     
    One last job - my website uses a cache system - by that, I mean if a query has already been run, don't run it again - look in the cache directory and use that html page. However, the cache page needs to be removed when you change data otherwise the changes don't show up.
     
    Job done? Not quite - I can now happily view the data for the class 23 and class 25 locomotives of the L&Y on my home computer, but, as of yet, you can't - at some time in the near(ish) future, I need to do a major upgrade and load these changes, and all the others I have done since March to the live server.
     
    So, that has taken me about 6 or 7 hours to execute - including research, scanning, parsing, correcting and loading data, just for one class of loco that most people will never look at! Is it worth it? I think so, and one day, I will have every locomotive that ran on Britain's public standard gauge rail network on the database. So, you can see the scope for errors to creep in - that's why I need eagle eyed observers and visitors to spot and report things.
     
    All the best for now.
×
×
  • Create New...