RMweb - Change of hosting, missing images - April 2022

[RedgateModels]

8 hours ago, AY Mod said:

 

That was what I resigned myself to in a wider form but I carried on prodding and poking people to make sure I wasn't letting a hard work solution go without investigation. However, I realistically knew some time back that a hoped-for outcome wouldn't be even remotely likely so my approach, which may seem poor on one level, was to let the dust settle and let everyone adjust to the position without causing more upset over the problem than necessary. It wasn't to sweep it under the carpet but to stabilise the situation to make sure we'd all, or as many as possible, would carry on communicating together.

An approach that I suspected was the case and which you judged right 🙂

[Colin_McLeod]

Thanks @Andy Y for youre efforts and the update. It's unfortunate that the pictures are not recoverable from the back up but it's not the end of the world. 

I'll make a start on reloading pictures where relevant/useful to  railway modelling but I'm not going to bother with 'passing amusement' posts or threads such as Whacky Signs and the Jokes thread. 

[St Enodoc]

10 hours ago, martin_wynne said:

 

RMwebbers looking for missing photos may find them on the Wayback Machine (Internet Archive). Unfortunately the change in URL format makes searching the Wayback Machine tricky and time-consuming.

 

But many of the missing photos are available on there, and could be copied back into topics. For example on this page:

 

 https://www.rmweb.co.uk/topic/15931-bury-thorn-sons/page/2/

 

this post has no attached image showing currently on RMweb:

 

 

 

But on the Wayback Machine the attached image is showing and could be copied back:

 

 

See: https://web.archive.org/web/20120502091157/http://www.rmweb.co.uk/community/index.php/topic/15931-bury-thorn-sons/page__st__25

 

Here is a direct link to that image, proving it could be downloaded and copied back into the original post:

 

 

There are many more such images on the Wayback Machine, but finding them is time-consuming. This link has a list has over 200 pages of links to topic content archived on the Wayback Machine from RMweb:

 

https://web.archive.org/web/*/https://www.rmweb.co.uk/community/index.php/topic/*

 

p.s. That page says "More than 10,000 URLs have been captured for this URL prefix."

 

cheers,

 

Martin.

That's very useful, Martin - thanks. I will try it for some photos in posts made by former RMwebbers who are no longer with us.

[Matt C]

14 hours ago, St Enodoc said:

That's very useful, Martin - thanks. I will try it for some photos in posts made by former RMwebbers who are no longer with us.

 

The only problem I forsee is that you wouldn't have access to those posts to re upload the Images. ?

@AY Mod      would have to set up some special/ temporary permissions to allow you or any other member prepared to carry out such work, to be able to edit other members/ ex members posts ?

[AY Mod]

15 minutes ago, Matt C said:

 

The only problem I forsee is that you wouldn't have access to those posts to re upload the Images. ?

@AY Mod      would have to set up some special/ temporary permissions to allow you or any other member prepared to carry out such work, to be able to edit other members/ ex members posts ?

 

It's not ideal (because the chronology would be warped) but the original post could be quoted and the images uploaded into an edit of the quote.

[Harlequin]

@AY Mod would it be possible to provide a mechanism for members to inject images directly into the database? Effectively allowing us to manually restore images from whatever external backup sources we have.

 

Doing that, and assuming the filenames are identical to the originals, should then make the old posts magically spring to life just like an internal backup restore.

 

In fact, it would be possible to write a script that hacks through the Wayback machine, doing that automatically...

 

[AY Mod]

8 minutes ago, Harlequin said:

@AY Mod would it be possible to provide a mechanism for members to inject images directly into the database? Effectively allowing us to manually restore images from whatever external backup sources we have.

 

Doing that, and assuming the filenames are identical to the originals, should then make the old posts magically spring to life just like an internal backup restore.

 

In fact, it would be possible to write a script that hacks through the Wayback machine, doing that automatically...

 

 

Sadly not with the way Invision's platform is structured;: it's all via the software, I can't even FTP files onto the storage.

[BroadLeaves]

33 minutes ago, Harlequin said:

would it be possible to provide a mechanism for members to inject images directly into the database?

Can we use https://invisioncommunity.com/developers/rest-api

For example, https://invisioncommunity.com/developers/rest-api?endpoint=forums/posts/POSTitem lets you edit a post.

[AY Mod]

9 minutes ago, BroadLeaves said:

Can we use https://invisioncommunity.com/developers/rest-api

 

No, certainly not for users. I'm not managing and maintaining that.

[Ian J.]

Allowing general users to inject anything into a database is a really bad idea. It's explicitly prevented for so many security reasons on pretty much any RDBMS and/or NoSQL system. The only way to allow users to re-upload content would be to develop an interface to check user input to ensure it complies with what the database expects and weeds out the inevitable malicious attempts to misuse it. Which, funnily enough, is pretty much what we have already.

[Harlequin]

18 minutes ago, Ian J. said:

Allowing general users to inject anything into a database is a really bad idea. It's explicitly prevented for so many security reasons on pretty much any RDBMS and/or NoSQL system. The only way to allow users to re-upload content would be to develop an interface to check user input to ensure it complies with what the database expects and weeds out the inevitable malicious attempts to misuse it. Which, funnily enough, is pretty much what we have already.

 

I was enquiring whether it could be done at all as the first step. If so then obviously security would be the next step. If there was a way for a script to find the missing images in an external source, such as Wayback Machine, and post them into the forum system then users wouldn't need to be involved. Only the script itself would need authority.

 

The REST API clearly has the security aspect covered but it's not immediately clear whether it can do the image replacement/injection. There doesn't seem to be a direct way but reading and editing posts might work, even though that would be a bit clunky.

 

[St Enodoc]

8 hours ago, Matt C said:

 

The only problem I forsee is that you wouldn't have access to those posts to re upload the Images. ?

@AY Mod      would have to set up some special/ temporary permissions to allow you or any other member prepared to carry out such work, to be able to edit other members/ ex members posts ?

Ah, this is where my theory falls to the ground.

 

A. Elk (Miss)

[kevinlms]

On 02/09/2022 at 20:37, AY Mod said:

It is evident from matters over the preceding weeks that no further progress with the restoration of images is likely to be made. The previous hosts maintained that their earlier backup was viable but the majority of folders containing the images from each month over the years were empty or corrupted. Beyond initial good progress it's been a very small minority that were recoverable and restored.

 

I fully understand the frustrations on an individual basis, especially for those who had uploaded content and I can assure you no-one has been more annoyed about it than me. I can only apologise (although not having caused it) but I am grateful to and impressed by those who have made inroads into restoring content from their locally saved files.

 

It's not a good outcome but thanks to measures taken before the crash at the end of March is a whole lot better than it could have been.

Is it my imagination, or have posts with photos that were there recently, now appear to have disappeared?

 

An example being the opening page here. I'm sure the hut photo has been there recently.

 

 

[AY Mod]

59 minutes ago, kevinlms said:

Is it my imagination

 

Probably yes.

[Re6/6]

On 02/09/2022 at 12:33, NHY 581 said:

 

 

Thanks Andy. 

 

I for one fully understand the efforts you have gone to over these last months and how difficult things have been. Your annoyance though is completely understandable given the manner in which you look after us. All too often you go over and above that which could reasonably be expected of you. 

 

However, no apology is needed or expected from yourself. On the contrary, given the nature of some comments posted by certain members, it is they who should be apologising to you. 

 

As I have maintained, no fingers should be pointed in any  direction other than the company entrusted with looking after RMweb, whose name shall not be spoken. 

 

Reaching this point and drawing this conclusion, was, sadly,  perhaps inevitable but as you so rightly say, it could have been worse. 

 

Now we have this confirmation, members can now simply go about the business of re-populating their respective threads as best they can and to the extent they feel is appropriate. 

 

That is all that is left to do now so let's just get on with it. 

 

Thank you once again. 

 

Rob. 

 

 

 

I wholeheartedly agree with all that Rob says. Re-populating threads will be a lengthy task probably best suited for the winter months (if I can find the images/remember what they were....!).

[Dave 46]

With regard to comments on page 43 regarding personal copies of threads, I have, in the past, copied each page by using "save pages as" from the drop down menu (Firefox).  A suitable file name should be chosen, with the page number, and this will save the HTML code for each page together with a folder(of the same name) with all the ancilliary files.  By opening the HTML file (offline) the page will load with Photos.

I did have one problem (still have it).  I saved the Silhouette thread, and the first batch of pages saved Feb 2017 suffer from threequarters of the page being blank, but those saved Oct 2017 are OK.

 

Dave

 

[Ben Alder]

6 hours ago, Dave 46 said:

With regard to comments on page 43 regarding personal copies of threads, I have, in the past, copied each page by using "save pages as" from the drop down menu (Firefox).  A suitable file name should be chosen, with the page number, and this will save the HTML code for each page together with a folder(of the same name) with all the ancilliary files.  By opening the HTML file (offline) the page will load with Photos.

I did have one problem (still have it).  I saved the Silhouette thread, and the first batch of pages saved Feb 2017 suffer from threequarters of the page being blank, but those saved Oct 2017 are OK.

 

Dave

 

Very interesting for my techno dinosaur mind,  thank you. What format do you save the pages as?

[Dave 46]

I just use the default format that Firefox (and I assume othrer browsers) use.  As I said above, this gives an HTML file together with a folder which the HTML file uses to load the images.  Just double click on the HTML and the page displays.  I can't remember but I assume the images are in the associated folder in .jpg or other suitable format.

 

Dave

 

[Jaggzuk]

A useful and simple tool to help find files on a PC.  Very fast and effective.  Making the job of finding photos to re-uplaod a bit easier, I for one have stored them all over the palce in folders.

 

https://everything.en.softonic.com/

[Barnaby]

Hello.

While searching the www for some information I was directed back to here, [seems many modelling roads lead back to RMweb].  Unfortunately, the very important part of this information a photograph was missing.

Where the photo should be was something like this>>> post-2973-0-68031900-1338290280_thumb.jpg and a little white box.  Is there any way I can get these details to resolve into the photo it previously had displayed.

It does seem that this info is directly related to the photo but I don't know how to resolve it.

If there is a way, can it be made into a sticky for future ref.

 

Thanks

 

Edited October 29, 2022 by Barnaby

[Ian J.]

12 minutes ago, Barnaby said:

Hello.

While searching the www for some information I was directed back to here, [seems many modelling roads lead back to RMweb].  Unfortunately, the very important part of this information a photograph was missing.

Where the photo should be was something like this>>> post-2973-0-68031900-1338290280_thumb.jpg and a little white box.  Is there any way I can get these details to resolve into the photo it previously had displayed.

It does seem that this info is directly related to the photo but I don't know how to resolve it.

If there is a way, can it be made into a sticky for future ref.

 

Thanks

 

 

RMweb had issues earlier in the year where images were lost by the internet hosting company of that time. The provider has since been changed.

 

Unfortunately it has not been possible to recover the images in their entirety and many are now missing. It will be up to the original creators of the threads, posts, blogs, etc, to re-upload their images. In some cases this will not happen as the creators are no longer on RMweb, either through leaving the site or passing away, or the creators no longer have the images.

 

Your first position on trying to see a lost image will be to message the creator of the relevant post and see if they are still around, and if so would they be willing to re-upload the image, if they still have it.

Edited October 29, 2022 by Ian J.

[Barnaby]

Thanks Ian J, I was hoping that a member who's much more adept at PC innards may have found a "tweak" that would allow me to do the hard graft for each missing photo but would bring them back to life.

I'll go and have a look back through the WAY BACK MACHINE. 

 

Best

[Kylestrome]

7 hours ago, Barnaby said:

I was hoping that a member who's much more adept at PC innards may have found a "tweak" that would allow me to do the hard graft for each missing photo but would bring them back to life.

 

If you don't own the original photos there is no way to access them. In other words, there are no magic "tweaks" no matter how gifted you are at using a computer.

 

 

Edited October 29, 2022 by Kylestrome

[Jon Fitness]

Have any more pictures gone missing since the big crash? I'm not sure but I think some of my more recent additions have disappeared...

JF

[AY Mod]

8 minutes ago, Jon Fitness said:

Have any more pictures gone missing since the big crash? I'm not sure but I think some of my more recent additions have disappeared...

JF

 

They shouldn't have but if you spot anything specific do let me know.