Category: Uncategorized (page 1 of 4)

Backup and transfer – part 2


Exporting the WordPress database and importing it into a new folder.

Exporting the WordPress files – takes a VERY long time due to the size of the site!

Haven’t got as far as importing the files into the new folder yet – may try exporting to DropBox first.


Following the instructions on

Downloaded WordPress files using Filezilla FTP.
wp-content is the VERY big file, taking hours, because of the photos.
It takes more than 3 hours to download 3451 photos (and the rest – see below!)

Exported WordPress database

Created new database
Created new database user
Created new password

Edited the wp-config.php file to match database, user and password

Attempted to import the database, got an error:



SQL query:

— Dumping data for table `wp_comments`

INSERT INTO `wp_comments` (`comment_ID`, `comment_post_ID`, `comment_author`, `comment_author_email`, `comment_author_url`, `comment_author_IP`, `comment_date`, `comment_date_gmt`, `comment_content`, `comment_karma`, `comment_approved`, `comment_agent`, `comment_type`, `comment_parent`, `user_id`) VALUES
(1, 1, ‘Mr WordPress’, ”, ‘’, ”, ‘2010-08-17 09:48:13’, ‘2010-08-17 14:48:13’, ‘Hi, this is a comment.<br />To delete a comment, just log in and view the post&#039;s comments. There you will have the option to edit or delete them.’, 0, ‘1’, ”, ”, 0, 0),
(6, 151, ‘drieniincanny’, ‘’, ”, ‘’, ‘2010-11-03 07:52:39’, ‘2010-11-03 07:52:39’, ‘<a href=”″ rel=”nofollow”>levitra levitra</a>\r\n<a href=”″ rel=”nofollow”>60 forum levitra 86</a>\r\n<a href=”″ rel=”nofollow”[…]

MySQL said: Documentation
#1062 – Duplicate entry ‘7’ for key ‘PRIMARY’


Looked at localhost.sql file – found lots of junk comments that don’t show in WordPress. But they were from another WordPress site of mine – in fact it was a dump of the wrong database!


Trying again…

Deleted swehs2 database

Created new database paulhul1_swehs2
Didn’t need to create new database user and password, already created
Added the new user to database paulhul1_swehs2, gave it all privileges

Had already edited the wp-config.php file to match database, user and password

About to import the database using phpmyadmin – but site started being maintained! came back online, tried to import the database into paulhul1_swehs2 using phpmyadmin
Response:  “Import has been successfully finished” – Success!



At the six hour mark, the WordPress files download was still going, though it’s moved on from photos to plugins like JetPack.

I wonder whether the Duplicator plugin could exclude the photos, and deal with them via FTP.

Or speed things up by Duplicator backing up to DropBox? Need to check the total size of the download. Uncompressed it’s going to be in excess of 5GB. But I have a terabyte of paid-for space! However, would my host just time out?

A useful-looking Dup;licator writeup is here:

And it’s stopped – more than six hours for wp-content. Finishing off with wp-includes and wp-snapshots. WiFi is on max 100 kbs. Maybe I need to cable up to the router. Needs at least a 12 metre Ethernet cable.

Maplins 20m Red Cat6 UTP Network Patch Cable £29.99, Code: A30UH

Strangely Filezilla shows that it’s copying some files to G:\swehsprototype\histpics\….jpg – but they don’t show up on the G: drive in File Explorer. The remote names are /tactive/histpics/….jpg and /bpics/….jpg. I wonder where those are in the file structure? Sounds like something copied from Marcus’s site.

The wp-snapshots folder takes ages to download as well – it looks like this might be the Duplicator package repository.
I don’t think I needed to back these packages up
– is there anything else in the folder except packages and installer?

Wifi has now speeded up to between 3 and 15 Mbps – that’s better!



Connected the PC to the router via cable – very much faster!

(Though my PC had a DNS problem with the Wifi after I disconnected it)

Backup and transfer – part 1

Summary – the automated backup system no longer works because of the size of the data, particularly the photos.


Tried Duplicator plugin – see video at:
Warning that large files and total size may cause timeouts

Build Interrupt
The current build has experienced an issue.
Please try the process again.
Server Status: 405 -Not Allowed
Error Message:
405 Not Allowed
Build Folder: /home4/paulhul1/public_html/swehs/wp-snapshots/tmp

  • Note: This is visible on FileZilla as just  /swehs/wp-snapshots/tmp/

Some servers close connections quickly; yet the build can continue to run in the background. To validate if a build is still running; open the ‘tmp’ folder above and see if the archive file is growing in size. If it is not then your server has strict timeout constraints. Please visit the support page for additional resources.

Looked in the ‘tmp’ folder – archive file was not growing in size, therefore unsuccessful.

Did “Try again” – same result.
Detailed explanation here:

Next step will be to try a manual backup and migration – see:

May be best to:

  1. Trial manual download
  2. Trial upload to a duplicate folder “/wphs” on Paul’s hosting (Would need to create a new MySQL database)
  3. Buy new hosting and set up new Society name and URL(s) e.g., then upload there

Histelec News and Supplements 63 and 64 added

Histelec News 63 and 64 added
Supplements 63 and 64 added

2017 events added and minor updates

2017 events added to events diary – to be updated again after 14 Feb committee meeting.

MOJO Marketplace plugin updated.

Twenty Thirteen and Twenty Twelve themes updated (though this site actually uses Hemingway theme)

Adding CEGB and CEB photo folders

All 11 CEGB folders – around 800 photos – have now been imported

Previously imported
SWEHS 5.1 CEB & CEGB Oakfield Road Bristol

Imported 28/10/2016
SWEHS 80.0 CEGB Pembroke Icing
SWEHS 81.0 CEGB Power Stations
SWEHS 82.0 CEGB Final Days of Portishead Power Station

For some reason no data was imported for 82.0 – should be in Table 4.
Uploaded Table 4 again using Filezilla, ran Import again, data uploaded.
Concluded that uploaded version of CSV may have been corrupt.

This is useful for any future instance of “no data” if the data is actually in the CSV.

Imported 31/10/2016
SWEHS 87.0 CEGB Durley Park
SWEHS 88.0 CEGB Overhead Lines and Projects
SWEHS 89.0 CEGB Miscellaneous
SWEHS 90.0 CEGB Severn Cable Tunnel
SWEHS 91.0 CEGB Steel Tower Lines Post Nationalisation
SWEHS 83.0 CEGB and SWEB Bulk Supply Points
SWEHS 85.0 CEGB Two Tier Project 1985

Outstanding issues

Updated 31/10/2016

Recent progress:

  • Recent work has concentrated on the photos.
  • Albums set up and photos successfully uploaded for all sections of South West before nationalisation, plus CEB and CEGB Oakfield Road.
  • There are now 2703 photos in the system, plus 127 “test” photos in Photos Album. 1133 of these are the Artefacts.
  • All of these are searchable (but are made searchable by a rather tedious manual process – no way round it)
  • All Supplements are now complete.
    Only outstanding Histelec News are nos. 1 to 10, 12, 13 – awaiting scanned PDFs.
  • 4 of the 11 CEGB photo folders have been uploaded, the other 7 have been set up as empty albums.
  • (Dummy) menu entry for Generating Stations now includes “renewal energy” in title because we have a good collection of hydro stations and wind turbines.

Strategic issues:

  • There are so many photos and so much data in the website now that I need to turn to Backup and Restore functionality, together with how to migrate the whole site to another host. Also see for WPPA-specific issues.
  • 7 more CEGB photo folders to upload (around 750 photos) – should be routine, though time-consuming.
  • Before the remaining photos can be uploaded …..
  • Then there are a further 78 folders of photos to upload – discussing with DC how to structure them. The 78 folders contain around 5850 SWEB photos, plus around 240 Sox Hosegood Collection helicopter photos, and around 130 other photos.

General issues:

  • I’ll check in due course which items I show that haven’t got descriptions, maybe where a photo has been uploaded but with no description as yet. But see discovery made under “Adding CEGB and CEB photo folders”
  • Also need to decide how to deal with PDF documents as the Photo plugin won’t handle these.
  • Improving the slideshow displays to make the Previous and Next buttons obvious, etc (see John F email 28/10/2016) – most recently I’ve ticked Photo Album Tables II B (slide shows) settings 13.1 and 13.2
  • Possibly remove the word nationalisation from 5.2, 7.0, 9.0, 15.0 & 20.0 to avoid duplication? The point of including “before nationalisation” was to avoid a confusing duplicate name for the equivalent after nationalisation – e.g. having two albums called “Bath”. Having said that, at the moment I’m using menus that are automatically generated from the album names and parent/child structure. If it’s the menu entries that are a concern, rather than the actual album names, , I should be able to do this manually when I’ve got all the photos loaded.
  • Any other comments from Committee members

Other sections of old website yet to be tackled:

  • Contacts page – just need to decide what we put on this page
  • Archives – needs updating – existing content was previous “cabinet listing” carried over from old website – have asked PGL and JG whether they could do this
  • There’s also another Archives page of Articles and Speakers in Study Corner – document just needs updating
  • Generating Stations – five pages or so – could just copy the content over
  • Films – only 3 videos copied over so far,  need to upload in a similar way to photos – or should I create pages with links? Note that there is a “Videos” post type – is there a “Videos” page type?
  • Films – manual text in Description field – would this be better for search?
  • Histelec News – last few very early issues to be uploaded when scanned
  • Web links – a simple page of links – need to check that they’re all still active
  • Days Out – probably needs a major rewrite. Low priority.
  • Museum – link needed for Curator – ties up with Contacts page.
  • Tramways – this section is very different. Need to find where the documents are stored and reformat them. Includes miniature and cliff railways.
  • Artefacts – needs a descriptive page like the old website, i.e. instructions on how to use the search and view facilities.
  • Photos – need a similar descriptive page of instructions. Maybe Films also?

Detailed Artefacts and Photos issues:

  • Many near-duplicate pictures when one per artefact would usually suffice – will ask DC to review and select

Additional future content

  • While those who have worked/are working the electricity industry and specifically SWEB will be aware of its operational structure and how it has changed over a number of years, this may not be so for others. Would be worthwhile including a a short glossary covering pre nationalisation, nationalisation and privatisation structures with associated dates and possibly a map/s. Perhaps a “family tree” also? It could link to Histelec News & Articles items for greater detail where these aspects have been covered. HAve asked PGL and JG whether they could do this

Issues updated up to 31/10/2016:

  • Investigate producing all the Archive Photographic Register Table CSVs and combining them. Then this could be run against each batch of photos uploaded to an Album. Not necessary 28/10/2016
  • Actually, it may not be necessary to combine the files. Maybe the Import will handle multiple CSV files? Need to test out with 2 or 3. Yes it does, proved 21/09/2016
  • The photo search facility is only searching the test “Photos” album, not the others. Need to find out how to amend this. Sorted by copying relevant code into each photo’s Description field 21/09/2016
  • Not sure where the Home Page slideshow is getting its photos from – this may need changing
    It’s a WordPress gallery, not WPPA functionality – Code is on Welcome page, just uses nominated picture numbers. Found 21/09/2016
  • There are 2 more Registers to convert to CSV – Registers 10 and 11 did not bring through proper photo filenames
    Turns out that they are duplicates, both deleted.
  • DC sent me SWEHS000392.jpg which is one of the Tinsley units – but there are already 000392a, b and c, the meter with the ball-bearings. Should the Tinsley unit be a different filename? Also 000392 is labelled as the Tinsley oscillator, which is 000538. 000392 is the Tinsley power supply unit. I will be able to straighten the Tinsley photos out by swapping the filenames and uploading them again with the new batch of Artefacts photos Tinsley entries edited 31/10/2016


Photo data cleaning, converting more Registers, more photo uploads


Minor data issues dealt with, but Registers 5 to 9 now converted to CSV and uploaded. Registers 10 and 11 did not bring through proper photo filenames so left for now.

Remaining “South West before nationalisation” Albums loaded complete with data. So we’re up to 1949 now.

CEGB and CEB folders all added.

Around 4000 photos in the system (as at 31/10/2016), of which 1133 are the artefacts.

Data issues

Discovered that some lines, even in Registers already converted, have multiple filenames. Lines split into individual filenames.

Converted Registers 5 to 9 to CSV.

Registers 10 and 11 deleted – they were duplicates with proper photo filenames.

Noticed that some filenames in Registers 3, 8 and 9 have blanks before the “.jpg” – checked one batch, and they really have! Leaving this issue for now, but need to bear in mind as a possible cause if photos are not matched with data during upload.

Uploading new data Tables

Uploaded Tables 1 to 9 CSVs to Depot using Filezilla.

Set default album to BCED in Import Photos
Went to Import Photos

Response: 9 CSVs imported, 162 photos processed. 7597 photos skipped.
Are these 162 photos that had no data?

Went to Album Admin, BCED, Edit
Couldn’t see any without data but didn’t look right through

Noticed an error in SWEHS 3.5.384.jpg “Stuff Mills” instead of “Snuff Mills”
Amended in Table 1.csv
Copied Table 1 again to Depot (using Filezilla) as a test of updating
Ran Import
Response: 9 CSVs imported, 460 photos processed. 2850 photos skipped.

Went to Album – checked, and it had updated to “Snuff Mills”. Success! This means that data amendments can be done by re-uploading and importing the CSV files, without having to reload the photos temselves.

More photo uploads

Previously – albums set up and photos successfully uploaded for BCED, Bristol Tramways and Carriage, CEB and CEGB Oakfield Road Bristol, Bath, Cornwall, Cornwall EWA Edmonds Collection.

Set up remaining “South West before nationalisation” Albums:
SWEHS 13.0 North Somerset Electricity Supply Co
SWEHS 16.0 Bridport Bridgwater Lyme Regis Minehead Wellington
SWEHS 17.0 Taunton
SWEHS 20.0 Weston Super Mare
SWEHS 9.0 Devon
SWEHS 11.0 Exeter
SWEHS 15.0 Plymouth
SWEHS 18.1 to 18.3 Torquay Borough Electricity Department

Used Filezilla to upload each set of photos in turn. Set each Default Album as appropriate, Import.

In Album Admin, Edit, set each Description as follows (that’s the tedious bit, needed for searchable indexing)



Noted that Weston before nationalisation contains some SWEB photos e.g. construction of Locking Road building.

That completes uploading the “Before nationalisation” photos – 3234 photos in the system, of which 1133 are artefacts.



Setting up Parent and Child Albums

Created new Album “South West Before Nationalisation”
Set Bath, BCED, Bristol Tramway, and both Cornwall albums to see this as their parent.

This Album has no other content – it does not have photos of its own (although it could have other content, if desired for any reason)

Option 1
Created new page “South West Before Nationalisation” to display this Album
Put it on the menu under Photos page.

This option displays the Album contents within the SW Before Nationalisation page.

Option 2

If preferred for a more compact display, I can just use HTML to display Album names and links with a thumbnail. Tried a rough version of this on a new page “South West Before Nationalisation version 2” (again, on the menu under Photos page)

This second version sends you to a page created for each album – playing with displays on the Bath page, but the different codes don’t seem to make any real difference.


Either option appears to be a good way for users to explore the various photo Albums by browsing through a heirarchy, as an alternative to a search.

Need to be clear what the effect is of allocating a parent Album – this powers the automatic display of Option 1, but doesn’t make any difference to the manual display of Option 2.


Experimenting with Search and Description


The search, and the display of the search results, are produced using the code in each individual photo’s Description field. There is no way to update many photos at once, it has to be done manually using the Album Admin – Bulk facility, copying and pasting on each line. Tedious but I managed to get up to 1000 or so updates an hour.

There are so many photos and so much data in the website now that I need to turn to Backup and Restore functionality, together with how to migrate the whole site to another host.

Films Album not edited – the manual descriptions might be better – consider **********.


I used the following code in the Description field in the Photos album:


Pasted this into the Description field on SWEHS 4.001.jpg.jpg, the first photo in Bristol Tramways and Carriage
Should now show up on Search for Counterslip.
Yes it does!

Second test: Pasted this into the Description field on SWEHS000001a.jpg, , the second photo in Artefacts.
Should now show up on Search for earthenware.
Yes it does!

So I need to bulk update the Description field in this Album.

Opened Table 2 CSV file
Created a column Description

Couldn’t put this in the lines:
Tried posting this into the cell for the first Counterslip photo, but it spread it out over various cells (because of the tr codes)

So copied the various custom cells into a Description_formula column I:
=F386&” “&A386&” “&B386&” “&C386&” “&D386&” “&E386&” “&G386&” “&H386
for the nine Counterslip photo records

Copied column I to Column J (Paste Values)
Saved as Archive Photographic Register Table 1 plus formula.csv
Deleted column I
Saved as Archive Photographic Register Table 1 plus description.csv
Close, Don’t save

Filezilla transferred to Depot and removed previous Table 1 CSV
Import Photos, Import
Response: “All available custom data fields are in use. There is no space for Description”
OK, let’s try something different:
In the Photo Albums -> Album admin -> Edit Album information -> Manage Photos admin page, you will see the input fields for the custom data right below the Description input area.
In the Album Description area, instead of “4.0 Bristol Tramways and Carriage” put:

Search for Counterslip didn’t work, so reverted to “4.0 Bristol Tramways and Carriage”

Created a new file Table 2 update.csv containing these two lines:
Filename, Description
Filename, Description
SWEHS 4.,<table><tbody><tr><td>w#cc1</td><td>w#cd1</td></tr><tr><td>w#cc2</td><td>w#cd2</td></tr><tr><td>w#cc3</td><td>w#cd3</td></tr><tr><td>w#cc4</td><td>w#cd4</td></tr><tr><td>w#cc5</td><td>w#cd5</td></tr><tr><td>w#cc6</td><td>w#cd6</td></tr><tr><td>w#cc7</td><td>w#cd7</td></tr></tbody></table>
… hoping that it would do a wildcard match to the Filename.

Import Photos, Import
Response: “All available custom data fields are in use. There is no space for Description”


So using a slower method:
Album Admin, Bulk, copy the code into Description for each photo individually.
Success! Search for tram returned 2, search for Counterslip returned 8, search for temple returned 1.

Updated all 385 photos in BCED Album in 25 minutes, so not too slow.
Again success! Search for cable returned 42, substation returned 66, transformer returned 11.

Did a similar manual update for all the other albums except Films.
Had a couple of “Comm error 405: Not Allowed” – moved to another page and back again, this time it worked.

**** This manual update will need to be done for each photo in each album uploaded ****

Films Album not edited – the manual descriptions might be better – consider **********.

Lots of photos and data in the website now – need to turn to Backup and Restore functionality / how to migrate to another host.

Converting databases via Excel to CSV files part 2


  • Register nos. 1 to 4 xlxs files converted to csv and uploaded
  • Albums set up and photos successfully uploaded for Bristol Tramways and Carriage, CEB and CEGB Oakfield Road Bristol, Bath, Cornwall, Cornwall EWA Edmonds Collection
  • There are now 2125 photos in the system, less 127 “test” photos in Photos Album. Just over half of these are the Artefacts.
  • There are 7 more Registers to convert to CSV
  • Then there are 98 more folders of photos to upload – need to discuss how to rationalise this number before I go any further. The 98 folders contain around 7500 photos.

Next steps:

  • Convert remaining Register xlxs files to csv and upload them
  • Run them against each of the existing Albums to fill in any data gaps
  • Rationalise Albums needed
  • Upload remaining photos, one Album at a time
  • Review what photos still have missing data – rectify manually if need be (ouch!)
  • Deal with issues documented on post “Outstanding issues from 18/09/2016”


The conversion process: (updated 26/10/2016)

  • Open Excel file – do not enable External Data Connections
  • Look for any multiple filenames in Column J “Photograph” and create duplicate lines, each with an individual filename
  • Copy columns M to T rows 1 and 2 from Table 2.xlsx into M1
  • Change M2 to formula to =J2

The contents of the two rows should now be:

file, id, item, location, date_made, description, comments, copyright

=J2, =A2, =B2&”_”&C2, =E2, =F2, =G2, =H2, =I2

  • Copy M2..T2 down to end of data
  • Highlight M1..T1 down to end of data and Copy
  • Highlight column V and paste values
  • Copy columns V..AC
  • Open new spreadsheet
  • Paste into A1
  • Save spreadsheet as CSV (MS-DOS) – “keep using that format”
  • Close CSV spreadsheet – do not save changes


Did all the above for first 4 tables, i.e. Registers 1 to 4.
The Registers each contain a real mix of photos – the best bet will be to load a batch of photos to an appropriate Album, and run against all the CSV files to index them.

Set up new Albums down as far as 7.1 Cornwall EWA Edmonds Collection
Used Filezilla to copy CSV files up to Table 4 into Depot
(Bristol Tramways and Carriage, CEB and CEGB Oakfield Road Bristol, Bath, Cornwall, Cornwall EWA Edmonds Collection)

So now on to uploading:


Used Filezilla to copy Bristol Tramways and Carriage photos into Depot
Noted that photos seem to be called .jpg.jpg (double suffix) – we’ll sort that out later if we need to.

Import Photos – Default album Bristol Tramways and carriage
Ticked “Remove from depot after successful import”
Unticked “Do not create duplicates”
25 photos, 4 CSV files
Clicked Import
Response “Done processing files. 25 single photos imported. 4 CSVs imported, 16 photos processed. 3498 photos skipped”
Guessing that only 16 had data attached – checked in Album Admin, yes that is the case

So it looks like we can use multiple CSV files at once with no problem – SUCCESS!

Repeated photo Filezilla for 5.1 CEB & CEGB Oakfield Road Bristol
Noted that photo 014 has filename .jpg (single suffix) but 015 has .jpg.jpg (double suffix) – check these as test cases
Changed default Directory, clicked Import

Response: “Done processing files. 4 CSVs imported, 20 photos processed. 4370 photos skipped”
But there were 40 photos, and they’ve all disappeared from Depot. However they’re all visible in Album Admin, justa few without data. No problem.
Photos 014 and 015 both loaded OK regardless of whether single or double suffix.

Repeated photo Filezilla for 5.2 Bath (21 photos), change default directory, Import.
Response: “Done processing files. 21 single photos imported. 4 CSVs imported, 21 photos processed. 4349 photos skipped”

Repeated photo Filezilla for 7.0 Cornwall (93 photos), change default directory (OR I THOUGHT I DID!), Import.
Response: “Time out. 53 photos imported. Please restart this operation” – Import again.
Response: “Done processing files. 4 CSVs imported, 0 photos processed. 4259 photos skipped.”
But all photos has disappeared from depot.

Album Admin – shows Cornwall as having no photos yet in Album. Gone to wrong Album?
Yes, found them in Bath Album. Album Admin – Bulk – Move was an easy solution.
Cornwall Album now shows all 93 photos.

Repeated photo Filezilla for 7.1 Cornwall EWA Edmonds Collection (210 photos), change default directory, Import.
Response: “Time out. 95 photos imported. Please restart this operation” – 115 left – Import again.
Response: “Time out. 80 photos imported. Please restart this operation” – 35 left – Import again.
Response: “Done processing. 35 single photos imported. 4 CSVs imported, 206 photos processed. 3990 photos skipped”
Discrepancy of 4 photos, assumed no data.


Next steps:

  • Convert remaining Register xlxs files to csv and upload them
  • Run them against each of the existing Albums to fill in any data gaps
  • Upload remaining photos, one Album at a time
  • Review what photos still have missing data – rectify manually if need be (ouch!)
Older posts

© 2017

Theme by Anders NorenUp ↑