Author: paul.hulbert@gmail.com (page 1 of 4)

Experiments with photo page layouts 2

Decided that the best bet – from a backup point of view – would be to store the data files such as photos, videos and documents externally (rather than in Media), and to keep the WordPress installation light so that it would back up easily.

Successfully trialled external photos, presented using a table layout. Altenate rows are photo, and label (with a white background). You need a blank header row to get the white background on the right rows. This presents well on phone and tablet as well.

It’s essential to have the right file permissions for this to work (755 on the folder, 644 on the files – may need to review the “Execute” permissions)

The experiments at http://www.paulhulbert.org/swehs/bath-test-photos-6/ were with fairly small files, 135 kb (900 x 600 px) to 2 Mb (3304 x 2397 px). An “average” one would be 740 kb (1632 x 1172). Need to consider resizing to a standard size. The photos are presented as 300 px wide, so resizing all to say 1500 px wide would be a good compromise – need to try using bulk resizing in Photoshop Elements, see https://digital-photography-school.com/batch-processing-in-photoshop-elements/

Followed this procedure, selecting 1500 px, Constrain Proportions, and 72 dpi, convert files to jpg Medium Quality. The result was 103 kb to 503 kb, all 1500 px wide. Worked well. Uploaded to replace files, and the page loads quickly. Quite good quality even when clicked on to magnify.

 

Experiments with photo page layouts

I played with a table layout, but one-column looked sparse, and two-column wouldn’t fit on a mobile screen.

However just dropping the photos in worked fine, if the photos were centred.

Then when I added some text – lorem ipsum paragraphs – I found that both text and photos fitted neatly on the mobile, as well as on iPad. And the portrait format on the mobile even formatted neatly when some of the photos were left or right aligned!

(See http://www.paulhulbert.org/swehs/bath-test-photos-3/ )

The vertical photos were resized to custom width 300.

The photos had Captions and Alt Text generated from a spreadsheet, e.g. SWEHS_5.2.021.jpg – Date 1920 – Customer record.

Note that the photos were renamed before being uploaded to avoid the problem of the space in the name – not sure that this was strictly necessary. Try again in the next trail.

Once I have a procedure, need to try a nearly-blank installation of WordPress with say 1000 photos loaded, and then whether I can then run a backup properly.

Header logo changed

Header logo changed to “Western Power Electricity Historical Society”, with explanation in text of Home Page.

Backup and transfer – part 2

Summary

Exporting the WordPress database and importing it into a new folder.

Exporting the WordPress files – takes a VERY long time due to the size of the site!

Haven’t got as far as importing the files into the new folder yet – may try exporting to DropBox first.

Details:

Following the instructions on http://www.wpexplorer.com/migrating-wordpress-website/

Downloaded WordPress files using Filezilla FTP.
wp-content is the VERY big file, taking hours, because of the photos.
It takes more than 3 hours to download 3451 photos (and the rest – see below!)

Exported WordPress database

On justhost.com
Created new database
Created new database user
Created new password

Edited the wp-config.php file to match database, user and password

Attempted to import the database, got an error:

=======================

Error

SQL query:


— Dumping data for table `wp_comments`

INSERT INTO `wp_comments` (`comment_ID`, `comment_post_ID`, `comment_author`, `comment_author_email`, `comment_author_url`, `comment_author_IP`, `comment_date`, `comment_date_gmt`, `comment_content`, `comment_karma`, `comment_approved`, `comment_agent`, `comment_type`, `comment_parent`, `user_id`) VALUES
(1, 1, ‘Mr WordPress’, ”, ‘http://wordpress.org/’, ”, ‘2010-08-17 09:48:13’, ‘2010-08-17 14:48:13’, ‘Hi, this is a comment.<br />To delete a comment, just log in and view the post&#039;s comments. There you will have the option to edit or delete them.’, 0, ‘1’, ”, ”, 0, 0),
(6, 151, ‘drieniincanny’, ‘fedasebr@mailnah.co.cc’, ”, ‘94.142.134.144’, ‘2010-11-03 07:52:39’, ‘2010-11-03 07:52:39’, ‘<a href=”http://profiles.friendster.com/122853886″ rel=”nofollow”>levitra levitra</a>\r\n<a href=”http://profiles.friendster.com/122854278″ rel=”nofollow”>60 forum levitra 86</a>\r\n<a href=”http://profiles.friendster.com/122854406″ rel=”nofollow”[…]

MySQL said: Documentation
#1062 – Duplicate entry ‘7’ for key ‘PRIMARY’

===================

Looked at localhost.sql file – found lots of junk comments that don’t show in WordPress. But they were from another WordPress site of mine – in fact it was a dump of the wrong database!

 

Trying again…

Deleted swehs2 database

On justhost.com
Created new database paulhul1_swehs2
Didn’t need to create new database user and password, already created
Added the new user to database paulhul1_swehs2, gave it all privileges

Had already edited the wp-config.php file to match database, user and password

About to import the database using phpmyadmin – but Justhost.com site started being maintained!

Justhost.com came back online, tried to import the database into paulhul1_swehs2 using phpmyadmin
Response:  “Import has been successfully finished” – Success!

 

————

At the six hour mark, the WordPress files download was still going, though it’s moved on from photos to plugins like JetPack.

I wonder whether the Duplicator plugin could exclude the photos, and deal with them via FTP.

Or speed things up by Duplicator backing up to DropBox? Need to check the total size of the download. Uncompressed it’s going to be in excess of 5GB. But I have a terabyte of paid-for space! However, would my host just time out?

A useful-looking Dup;licator writeup is here:
https://www.wpkube.com/move-backup-website-wordpress-duplicator-plugin/

And it’s stopped – more than six hours for wp-content. Finishing off with wp-includes and wp-snapshots. WiFi is on max 100 kbs. Maybe I need to cable up to the router. Needs at least a 12 metre Ethernet cable.

Maplins 20m Red Cat6 UTP Network Patch Cable £29.99, Code: A30UH

Strangely Filezilla shows that it’s copying some files to G:\swehsprototype\histpics\….jpg – but they don’t show up on the G: drive in File Explorer. The remote names are /tactive/histpics/….jpg and /bpics/….jpg. I wonder where those are in the file structure? Sounds like something copied from Marcus’s site.

The wp-snapshots folder takes ages to download as well – it looks like this might be the Duplicator package repository.
I don’t think I needed to back these packages up
– is there anything else in the folder except packages and installer?

Wifi has now speeded up to between 3 and 15 Mbps – that’s better!

==============

 

Connected the PC to the router via cable – very much faster!

(Though my PC had a DNS problem with the Wifi after I disconnected it)

Backup and transfer – part 1

Summary – the automated backup system no longer works because of the size of the data, particularly the photos.

Detail:

Tried Duplicator plugin – see video at:
https://www.youtube.com/watch?v=tdP3quWLM0Q
Warning that large files and total size may cause timeouts

Response:
Build Interrupt
The current build has experienced an issue.
Please try the process again.
Server Status: 405 -Not Allowed
Error Message:
405 Not Allowed
nginx/1.10.2
Build Folder: /home4/paulhul1/public_html/swehs/wp-snapshots/tmp

  • Note: This is visible on FileZilla as just  /swehs/wp-snapshots/tmp/

Some servers close connections quickly; yet the build can continue to run in the background. To validate if a build is still running; open the ‘tmp’ folder above and see if the archive file is growing in size. If it is not then your server has strict timeout constraints. Please visit the support page for additional resources.

Looked in the ‘tmp’ folder – archive file was not growing in size, therefore unsuccessful.

Did “Try again” – same result.
Detailed explanation here:
https://snapcreek.com/duplicator/docs/faqs-tech/#faq-trouble-100-q

Next step will be to try a manual backup and migration – see:
http://www.wpexplorer.com/migrating-wordpress-website/

May be best to:

  1. Trial manual download
  2. Trial upload to a duplicate folder “/wphs” on Paul’s hosting (Would need to create a new MySQL database)
  3. Buy new hosting and set up new Society name and URL(s) e.g. wpehs.org, then upload there

Histelec News and Supplements 63 and 64 added

Histelec News 63 and 64 added
Supplements 63 and 64 added

2017 events added and minor updates

2017 events added to events diary – to be updated again after 14 Feb committee meeting.

MOJO Marketplace plugin updated.

Twenty Thirteen and Twenty Twelve themes updated (though this site actually uses Hemingway theme)

Adding CEGB and CEB photo folders

All 11 CEGB folders – around 800 photos – have now been imported

Previously imported
SWEHS 5.1 CEB & CEGB Oakfield Road Bristol

Imported 28/10/2016
SWEHS 80.0 CEGB Pembroke Icing
SWEHS 81.0 CEGB Power Stations
SWEHS 82.0 CEGB Final Days of Portishead Power Station

For some reason no data was imported for 82.0 – should be in Table 4.
Uploaded Table 4 again using Filezilla, ran Import again, data uploaded.
Concluded that uploaded version of CSV may have been corrupt.

This is useful for any future instance of “no data” if the data is actually in the CSV.

Imported 31/10/2016
SWEHS 87.0 CEGB Durley Park
SWEHS 88.0 CEGB Overhead Lines and Projects
SWEHS 89.0 CEGB Miscellaneous
SWEHS 90.0 CEGB Severn Cable Tunnel
SWEHS 91.0 CEGB Steel Tower Lines Post Nationalisation
SWEHS 83.0 CEGB and SWEB Bulk Supply Points
SWEHS 85.0 CEGB Two Tier Project 1985

Outstanding issues

Updated 31/10/2016

Recent progress:

  • Recent work has concentrated on the photos.
  • Albums set up and photos successfully uploaded for all sections of South West before nationalisation, plus CEB and CEGB Oakfield Road.
  • There are now 2703 photos in the system, plus 127 “test” photos in Photos Album. 1133 of these are the Artefacts.
  • All of these are searchable (but are made searchable by a rather tedious manual process – no way round it)
  • All Supplements are now complete.
    Only outstanding Histelec News are nos. 1 to 10, 12, 13 – awaiting scanned PDFs.
  • 4 of the 11 CEGB photo folders have been uploaded, the other 7 have been set up as empty albums.
  • (Dummy) menu entry for Generating Stations now includes “renewal energy” in title because we have a good collection of hydro stations and wind turbines.

Strategic issues:

  • There are so many photos and so much data in the website now that I need to turn to Backup and Restore functionality, together with how to migrate the whole site to another host. Also see http://wppa.nl/docs-by-subject/moving-site/ for WPPA-specific issues.
  • 7 more CEGB photo folders to upload (around 750 photos) – should be routine, though time-consuming.
  • Before the remaining photos can be uploaded …..
  • Then there are a further 78 folders of photos to upload – discussing with DC how to structure them. The 78 folders contain around 5850 SWEB photos, plus around 240 Sox Hosegood Collection helicopter photos, and around 130 other photos.

General issues:

  • I’ll check in due course which items I show that haven’t got descriptions, maybe where a photo has been uploaded but with no description as yet. But see discovery made under “Adding CEGB and CEB photo folders”
  • Also need to decide how to deal with PDF documents as the Photo plugin won’t handle these.
  • Improving the slideshow displays to make the Previous and Next buttons obvious, etc (see John F email 28/10/2016) – most recently I’ve ticked Photo Album Tables II B (slide shows) settings 13.1 and 13.2
  • Possibly remove the word nationalisation from 5.2, 7.0, 9.0, 15.0 & 20.0 to avoid duplication? The point of including “before nationalisation” was to avoid a confusing duplicate name for the equivalent after nationalisation – e.g. having two albums called “Bath”. Having said that, at the moment I’m using menus that are automatically generated from the album names and parent/child structure. If it’s the menu entries that are a concern, rather than the actual album names, , I should be able to do this manually when I’ve got all the photos loaded.
  • Any other comments from Committee members

Other sections of old website yet to be tackled:

  • Contacts page – just need to decide what we put on this page
  • Archives – needs updating – existing content was previous “cabinet listing” carried over from old website – have asked PGL and JG whether they could do this
  • There’s also another Archives page of Articles and Speakers in Study Corner – document just needs updating
  • Generating Stations – five pages or so – could just copy the content over
  • Films – only 3 videos copied over so far,  need to upload in a similar way to photos – or should I create pages with links? Note that there is a “Videos” post type – is there a “Videos” page type?
  • Films – manual text in Description field – would this be better for search?
  • Histelec News – last few very early issues to be uploaded when scanned
  • Web links – a simple page of links – need to check that they’re all still active
  • Days Out – probably needs a major rewrite. Low priority.
  • Museum – link needed for Curator – ties up with Contacts page.
  • Tramways – this section is very different. Need to find where the documents are stored and reformat them. Includes miniature and cliff railways.
  • Artefacts – needs a descriptive page like the old website, i.e. instructions on how to use the search and view facilities.
  • Photos – need a similar descriptive page of instructions. Maybe Films also?

Detailed Artefacts and Photos issues:

  • Many near-duplicate pictures when one per artefact would usually suffice – will ask DC to review and select

Additional future content

  • While those who have worked/are working the electricity industry and specifically SWEB will be aware of its operational structure and how it has changed over a number of years, this may not be so for others. Would be worthwhile including a a short glossary covering pre nationalisation, nationalisation and privatisation structures with associated dates and possibly a map/s. Perhaps a “family tree” also? It could link to Histelec News & Articles items for greater detail where these aspects have been covered. HAve asked PGL and JG whether they could do this

Issues updated up to 31/10/2016:

  • Investigate producing all the Archive Photographic Register Table CSVs and combining them. Then this could be run against each batch of photos uploaded to an Album. Not necessary 28/10/2016
  • Actually, it may not be necessary to combine the files. Maybe the Import will handle multiple CSV files? Need to test out with 2 or 3. Yes it does, proved 21/09/2016
  • The photo search facility is only searching the test “Photos” album, not the others. Need to find out how to amend this. Sorted by copying relevant code into each photo’s Description field 21/09/2016
  • Not sure where the Home Page slideshow is getting its photos from – this may need changing
    It’s a WordPress gallery, not WPPA functionality – Code is on Welcome page, just uses nominated picture numbers. Found 21/09/2016
  • There are 2 more Registers to convert to CSV – Registers 10 and 11 did not bring through proper photo filenames
    Turns out that they are duplicates, both deleted.
  • DC sent me SWEHS000392.jpg which is one of the Tinsley units – but there are already 000392a, b and c, the meter with the ball-bearings. Should the Tinsley unit be a different filename? Also 000392 is labelled as the Tinsley oscillator, which is 000538. 000392 is the Tinsley power supply unit. I will be able to straighten the Tinsley photos out by swapping the filenames and uploading them again with the new batch of Artefacts photos Tinsley entries edited 31/10/2016

 

Photo data cleaning, converting more Registers, more photo uploads

Summary

Minor data issues dealt with, but Registers 5 to 9 now converted to CSV and uploaded. Registers 10 and 11 did not bring through proper photo filenames so left for now.

Remaining “South West before nationalisation” Albums loaded complete with data. So we’re up to 1949 now.

CEGB and CEB folders all added.

Around 4000 photos in the system (as at 31/10/2016), of which 1133 are the artefacts.

Data issues

Discovered that some lines, even in Registers already converted, have multiple filenames. Lines split into individual filenames.

Converted Registers 5 to 9 to CSV.

Registers 10 and 11 deleted – they were duplicates with proper photo filenames.

Noticed that some filenames in Registers 3, 8 and 9 have blanks before the “.jpg” – checked one batch, and they really have! Leaving this issue for now, but need to bear in mind as a possible cause if photos are not matched with data during upload.

Uploading new data Tables

Uploaded Tables 1 to 9 CSVs to Depot using Filezilla.

Set default album to BCED in Import Photos
Went to Import Photos
Import

Response: 9 CSVs imported, 162 photos processed. 7597 photos skipped.
Are these 162 photos that had no data?

Went to Album Admin, BCED, Edit
Couldn’t see any without data but didn’t look right through

Noticed an error in SWEHS 3.5.384.jpg “Stuff Mills” instead of “Snuff Mills”
Amended in Table 1.csv
Copied Table 1 again to Depot (using Filezilla) as a test of updating
Ran Import
Response: 9 CSVs imported, 460 photos processed. 2850 photos skipped.

Went to Album – checked, and it had updated to “Snuff Mills”. Success! This means that data amendments can be done by re-uploading and importing the CSV files, without having to reload the photos temselves.

More photo uploads

Previously – albums set up and photos successfully uploaded for BCED, Bristol Tramways and Carriage, CEB and CEGB Oakfield Road Bristol, Bath, Cornwall, Cornwall EWA Edmonds Collection.

Set up remaining “South West before nationalisation” Albums:
SWEHS 13.0 North Somerset Electricity Supply Co
SWEHS 16.0 Bridport Bridgwater Lyme Regis Minehead Wellington
SWEHS 17.0 Taunton
SWEHS 20.0 Weston Super Mare
SWEHS 9.0 Devon
SWEHS 11.0 Exeter
SWEHS 15.0 Plymouth
SWEHS 18.1 to 18.3 Torquay Borough Electricity Department

Used Filezilla to upload each set of photos in turn. Set each Default Album as appropriate, Import.

In Album Admin, Edit, set each Description as follows (that’s the tedious bit, needed for searchable indexing)

USE BULK EDIT TO UPDATE DESCRIPTION – IT IS MUCH MUCH FASTER:

<table><tbody>
<tr><td>w#cc1</td><td>w#cd1</td></tr>
<tr><td>w#cc2</td><td>w#cd2</td></tr>
<tr><td>w#cc3</td><td>w#cd3</td></tr>
<tr><td>w#cc4</td><td>w#cd4</td></tr>
<tr><td>w#cc5</td><td>w#cd5</td></tr>
<tr><td>w#cc6</td><td>w#cd6</td></tr>
<tr><td>w#cc7</td><td>w#cd7</td></tr>
</tbody></table>

Noted that Weston before nationalisation contains some SWEB photos e.g. construction of Locking Road building.

That completes uploading the “Before nationalisation” photos – 3234 photos in the system, of which 1133 are artefacts.

 

 

Older posts

© 2017

Theme by Anders NorenUp ↑