RMweb - Change of hosting, missing images - April 2022

[BroadLeaves]

12 hours ago, NHY 581 said:

Whilst I understand your frustration, once again, THE SITE did not 'allow' it to happen. 

 

The fault lies with the company entrusted with the data. 

 

It's no good repeatedly pointing fingers at Rmweb. 

There's a standard rule for backups, known as 3-2-1:

• There should be 3 copies of the data
• On 2 different media
• With 1 copy being held off site (and preferably off line)

My understanding is that the failure of the RAID5 array that was hosting both the site's working data and the backups caused a total data loss, and that there was no other reasonably current backup. Backing up solely to the same array breaks all three rules. I imagine we'll never know if Dediserve explicitly stated how backups were managed, or if it was just a

Warners: "Do you do backups?"
Dediserve: "Yes"
Warners: * ticks box and moves on *

 I just hope that all the suitably pertinent questions have been asked of IPS and, more importantly, they gave the right answers. Once bitten twice shy and all that. IPS allow downloads of full backups, so provided that option is exercised with a suitable frequency, it won't be a problem in the future.

[BroadLeaves]

On 10/08/2022 at 19:28, Phil Parker said:

Officially, still work in progress. Practically, if you want them back in your posts, you'll need to restore them yourself.

As this has been going on for quite literally months with no end in sight, is it time to give the current approach (whatever that is - I think it's been referred to as "reindexing") up as a bad job?

A serious suggestion / question:

Have you considered asking for assistance from the RMWeb users themselves? I don't mean in the sense that everyone uploads all their photos again, but in the sense of building something automated to restore the missing images. Given the wide range of skills and experiences that are available across the broad spectrum of the RMWeb community, there may be some suitable expertise available.

[Colin_McLeod]

3 minutes ago, BroadLeaves said:

As this has been going on for quite literally months with no end in sight, is it time to give the current approach (whatever that is - I think it's been referred to as "reindexing") up as a bad job?

 

Sadly, I think you have hit the nail on the head.

[jamie92208]

Having now done about 6000 out of 7000 images can I say that I don't think that automation would help. The reloading is totally dependent on the original poster still having the missing images on a device.  I've little problem going back to about 2014 when I became much more methodical in my indexing. The other factor that has helped is that scanning all my negatives and slides during lockdown meant that many of the images are easily found even if their file names bear little relationship to the name I used when I loaded them first time around.  The actual up,oadi g is quite quick, it's finding the images that takes the time. The remaing ones are mainly from more social threads such as Early Risers so I'll probably not bother with those.  However I will see if there are others that could be useful.

 

Jamie

[St Enodoc]

40 minutes ago, jamie92208 said:

The reloading is totally dependent on the original poster still having the missing images on a device.

Two different things, Jamie. What you say applies from June 2021 to March 2022, from which period all photos are lost. Before June 2021 the advice was that the photos would be reloaded automatically but these seems to have stopped.

[Harlequin]

2 hours ago, BroadLeaves said:

As this has been going on for quite literally months with no end in sight, is it time to give the current approach (whatever that is - I think it's been referred to as "reindexing") up as a bad job?
 

 

2 hours ago, Colin_McLeod said:

 

Sadly, I think you have hit the nail on the head.

 

I think @AY Mod said that there was some dispute still holding up the full restoration of images. I.e. It's not purely a technical issue. Until that dispute is resolved one way or another, we don't know whether older images will be restored automatically or not. Negotiations may be sensitive and Andy will tell us when he can, no doubt.

 

[BroadLeaves]

8 hours ago, Harlequin said:

some dispute still holding up the full restoration of images

If that's the case, then at least a definitive statement to that effect (even if it gives no details about the nature of the dispute or the parties involved) would be useful, so that people don't spend potentially days doing something manually if a working automated solution is in place, but is just waiting for the lawyers to get things sorted out.

 

[Andy Hayter]

That was already given by Andy and no further update has been made.  You should assume that the statement still holds sway.

 

Mind you will have to read through 40 odd pages to find the statement.

[kevinlms]

On 15/08/2022 at 20:09, Geep7 said:

What I feel sad about, and i'm sure the rest of the RMWeb team do to, is that it is looking more likely (although not absolutely certain) that we've possibly lost a lot of content that can never be replaced, due to the thread owners having passed on. I won't mention any specific member's names or threads, but there were a few threads that I bookmarked and referred to quite a lot, due to my own modelling interest's, which I know sadly, will never have the images replaced if they are completely lost.

 

Please do not take this post in any way as a criticism of the current image situation on RMWeb. I totally understand and support, Andy and the RMWeb's team in their handling of this situation. There is nothing more that can be done, other than wait. It's just my personal feeling, should the worst case appear to be the outcome of the website upheaval.

Don't forget that previous versions of RMweb have been lost along with ALL content. At one stage Martin Wynne tried to get members to volunteer to create a database of some of the old site and despite several requests there were few volunteers.

[Blandford1969]

On 16/08/2022 at 08:39, BroadLeaves said:

If that's the case, then at least a definitive statement to that effect (even if it gives no details about the nature of the dispute or the parties involved) would be useful, so that people don't spend potentially days doing something manually if a working automated solution is in place, but is just waiting for the lawyers to get things sorted out.

 

You really have to work on the basis the images are not coming back. Much as that is sad hassling for an update when there is not is a time killer for Andy. 

[Geep7]

10 hours ago, kevinlms said:

Don't forget that previous versions of RMweb have been lost along with ALL content. At one stage Martin Wynne tried to get members to volunteer to create a database of some of the old site and despite several requests there were few volunteers.

That's a very good point, and I had forgotten about that. I think I joined RMWeb not long after the second iteration of the site was enabled, with the Archive section still available.

 

It's also something of a reminder that not everything on the internet is as permanent as it may appear, and eventually most of what is on the internet will pass into history (unless saved on the Wayback Machine).

 

There is an 80's sci-fi film quote that would be most appropriate for this. I won't post it, as those who know, will know what it is, and for those who don't.... probably Rutger Hauer's finest performance.

[john new]

On 15/08/2022 at 19:56, john new said:

Time to bite the bullet and start the steady plod of reinstatement of as many of my own images as I can find.

 

As others have posted, the really sad thing is that IF this restitution process does not happen, and it is looking unlikely now, then images from inspirational posters who are no longer with us can never be reloaded. 

Some more done today, all the definitely lost from the server images from June 2021 through to end March 2022 are now reinstated. What is not helping is that even though I had a fairly sensible naming policy for the resized RM web versions until recently I left them with the original full size versions not placed into an RM Web labeled graphics up-loaded folder as I do now.  

 

Let us just hope the legal issue/dispute (If that is what it is) preventing the pre June 2021 images being reloaded does eventually get resolved. Finding those older images is proving a slow process.

 

Edited August 19, 2022 by john new

[MrWolf]

I've got about fifty pages reposted so far. Where I no longer have the image I have left the original information in place, just in case.

[ejstubbs]

On 16/08/2022 at 12:35, Andy Hayter said:

That was already given by Andy and no further update has been made.  You should assume that the statement still holds sway.

 

Mind you will have to read through 40 odd pages to find the statement.

 

Or use the forum's own "in the current thread" search option (like I did, and it worked just fine - though it doesn't seem to exist in the mobile version of the site).

 

 

[Barnaby]

Several of my missing posts have wording like this showing >>>

This XML file does not appear to have any style information associated with it. The document tree is shown below.

<Error>

<Code>NoSuchKey</Code>

<Message>The specified key does not exist.</Message>

<Key>y320084/monthly_2020_03/535690195_somethingforthelockdown..jpg.5c8ded7cae5d958b7dbaeffebc19a639.jpg</Key>

<RequestId>KQSFSKVJXXX1YCQF</RequestId>

<HostId>XrkteUKT1xXWtqRypJpVMFfG3Z+PdAUX1TRZXGTrE2LH8wKvMn+CiYcrcUrQXDo6EVBvNJBoQPY=</HostId>

</Error>

 

Is there anything I can do to to bring the post back into full view again.

[eastglosmog]

A curiosity I have noticed is that although I have been (usually) reinstating exactly the same file as the one lost, the size of the reinstated photo seems to vary from the previous one.  No idea why.

[Nile]

Could that be because of the way the current forum software compresses images?

[john new]

34 minutes ago, Nile said:

Could that be because of the way the current forum software compresses images?

Yes, IIRC there is a post from Andy made a few days ago saying the software behind the forum automatically compresses image sizes. Too many people, I admit hat has me included from time to time, forget to reduce the file size before posting. Result the software does it for us.

[Andy Hayter]

Probably no bad thing John.  Microsoft seem to be able to completely change the way in which picture can be resized with each new version and while I did find it some months ago, what I thought to be the route has disappeared with my recent posting.

 

 

[eastglosmog]

My photos posted on here are resized using the Microsoft Windows resizer package, so not surprised file compressor does something different on here!  Some sizes have gone up as well as some going down, seems to be quite random.  As an example, my photo of Hythe Goods Yard I posted in the Goods Yard ballast thread originally had a size of 477kB and is 974kB now.  The original file is 633kB.

Edited August 25, 2022 by eastglosmog
Add example

[BroadLeaves]

It depends what is meant by "compression". There's at least five ways to reduce the file size for an image:

1. Reduce the resolution (e.g. shrink a 1000x700 pixel image to a 500x350 pixel image, with suitable interpolation).
2. Remove non-visible information (EXIF data, colour spaces, layering information, embedded thumbnails etc.).
3. Reduce the colour depth, optionally optimising the colour palette (e.g. reduce a 24-bit colour image to an 8-bit colour image).
4. Increase the "lossiness" of a lossy compression algorithm. The image remains the same size in terms of pixels, but image quality degrades.
5. Apply a different compression algorithm. The PNG spec includes four different optional compression filters, and different filters will yield different compression ratios (and hence ultimate file sizes) based on the underlying image data. It's even possible to apply different filters to each scanline, further increasing the compression at the expense of increased resource usage for the actual compression operation.

Of course, it's also possible to apply several of the above at the same time!

TL;DR: Without knowing how RMWeb is compressing images and how a third party tool is compressing them, any comparison is going to be very difficult.

[martin_wynne]

31 minutes ago, BroadLeaves said:

It depends what is meant by "compression". There's at least five ways to reduce the file size for an image:

 

6. crop the image.

 

Often images are posted with vast amounts of sky or foreground. If it's a creative image that's fine. But if you are just trying to show some specific object, you can crop away the irrelevant parts of the image. Which reduces the file size significantly. It may also make it easier to see the object.

 

Typical camera image uploaded to RMweb (2382 KB):

 

 

 

What the user was actually trying to show (712 KB ):

 

 

Martin.

[MrWolf]

I tend to crop all of my images to the focus area, they upload quicker and I can fit more images per post.

[Donington Road]

4 hours ago, MrWolf said:

I tend to crop all of my images to the focus area, they upload quicker and I can fit more images per post.

 

Cropped images are fine when you want to highlight the subject within the original photo and they do indeed reduce the file size as they have removed all the surplus pixels.

Many people however want to post their full size photo and may not be aware of how to resize it so they can upload multiple photos in one post.

 

You can reduce image file sizes in many ways without having to resort to cropping using resizing software, many resizing options are inbuilt in image processing software such as Photoshop Elements, Affinity Photo, ADSee Photo Studio, etc.

I use some of those just mentioned for processing photos, but I still find they do not reduce the files size sufficiently for my needs.

 

My choice is the free Faststone Image Viewer https://www.faststone.org/FSViewerDetail.htm

There are so many options in that program to reduce a photographs file size that I never bother with the commercial software for doing it any more.

 

The original image of the Azuma is 6000x4000 pixels and the file size 10.6mb.  This was reduced in Faststone by applying 25% reduction which reduced to file size to 524kb.

The whole image is there below, no cropping, no adjustments to colour or anything else, just resized.

Faststone is rather an underated program IMO, yes , it is basic on photo editing but for resizing, batch renaming, watermarking, adding text, etc. it is very good and it also plays video files very well.

 

Edited August 25, 2022 by Donington Road

[martin_wynne]

Another option to reduce the file size without changing anything else is Google's Weppy file format. Save the image as a WEBP file instead of JPG.

 

JPG  - 1465 KB:

 

 

__________________________________________________________

 

WEBP - 1140 KB:

 

 

Martin.