For a change, rather than looking at the past, today’s post is rather topical, and looks at a subject that at first glance may seem irrelevant to the blog – Artificial Intelligence, or AI.
When I write a post for the blog, I try to apply a degree of intelligence. I use a wide range of sources – books (old and new), maps, national and local archives, old newspaper archives, libraries, academic resources, visits to the site that is the subject of a blog post, and hopefully, some knowledge of London built up over the years.
I aways try to find more than one reference to key information about a topic, and only use a limited number of reputable sources on the Internet, for example British History Online.
Fortunately, I make very few mistakes, and when I have, I am very grateful to readers who have pointed this out via a comment or email, so I can correct.
The numbers of visitors to the blog has grown considerably since I started in 2014. Most of these readers come via Google, but over the last few months, I have seen that the blog is being accessed by AI services such as ChatGPT and Google Gemini.
There is much noise about Artificial Intelligence. Depending on what you read, AI is either going to save the world, or take everyone’s jobs and destroy the world.
Most of the end user AI services such as ChatGPT, Google Gemini, Microsoft Copilot and Grok (X was Twitter) are conversational based systems using extremely large collections of data.
Conversational, in that they attempt to provide the user with the impression that they are talking to another human, and that you can have a conversation with them. The large collections of data that they use to build a response, are from anything they can search – websites, blogs, archives, photos, books etc.
As someone who has written getting on for 2.5 million words in the blog during the last eleven years, I wanted to know what impact AI would have on this form of writing? Was AI using information that I had created, how accurate AI could be, and whether AI could replace a blog as an information source, and if so, how accurate would be the response.
So I thought I would give it a try.
I used a few of the most high profile AI tools to ask a simple question about one of my favourite places.
Imagine you are walking from St. Katherine Docks to Wapping, and as you walk along St. Katherine’s Way, you find an intriguing set of river stairs – Alderman Stairs – and you want to know their history, so you ask a few AI tools “What is the history of Alderman Stairs”, (you can find my post on Alderman Stairs, here) and this is what I did, starting with:
ChatGPT
ChatGPT was one of the first readily available AI tools, with the first release in November 2022, and the tool is currently on the fourth version. Chat, the first part of the name should be self explanatory, GPT stands for Generative Pre-Trained Transformer, which basically means it captures information from the Internet in order to learn how to respond to a question.
You can try out ChatGPT by clicking here, where you are presented with a simple “What can I help with?”, to which I asked the following question:
ChatGPT provides the answer to your question in a response screen, but also allows you to copy the answer as text, so in theory, I could use the answer to write a blog post, so the following text is an exact copy of ChatGPT’s answer to the question “what is the history of Alderman Stairs?“:
“Alderman Stairs is a historic set of stairs located in the City of London, connecting Upper Thames Street to Puddle Dock and the River Thames. The stairs have a rich history, and their name, along with their structure, reflects the changing history of London.
Origins of the Name:
- Alderman Stairs is named after an alderman, a senior elected official in the City of London. In this case, the name likely honors one of the city’s aldermen who played a significant role in its development or public works in the area.
- The exact alderman behind the stairs’ naming is not fully documented, but it was common in the medieval period for prominent citizens to have public infrastructure named after them, especially in riverfront areas.
Development and Historical Use:
- The stairs have long been part of London’s waterfront, providing access from the land to the river. They were essential for the transport of goods and passengers during the times when river trade was crucial to London’s economy.
- The Thames served as a bustling trade route, and stairs like Alderman Stairs were crucial for loading and unloading cargo. Ships would dock at these stairs to transfer goods such as coal, timber, and other materials necessary for London’s growth.
The Changes Over Time:
- Medieval and Early Modern Period: Alderman Stairs likely existed in some form by the late medieval period when river transport was key to London’s development. Stairs and wharves were often maintained by wealthy and influential individuals or companies.
- Victorian Era: The industrialization of London in the 19th century, along with major changes in the River Thames, led to the construction of larger docks and piers. Over time, this reduced the prominence of stairs like Alderman Stairs for commercial activity.
- 20th Century: By the 20th century, many stairs and docks along the Thames had fallen into disuse due to the decline of river trade and the increasing use of road transport. Alderman Stairs, like many others, would have lost much of its original commercial purpose.
Modern Day:
- Today, Alderman Stairs is primarily of historical interest and is considered a heritage site, reflecting London’s long relationship with the River Thames. The stairs themselves are located near the London Bridge area, in a part of the city that has been significantly redeveloped.
- Though the area has been extensively modernized, the stairs remain as a reminder of London’s medieval and early modern history. They have been restored to some degree and are occasionally used for public access to the river.
The history of Alderman Stairs highlights London’s transformation from a city dominated by river trade to one marked by modern infrastructure and development. Despite these changes, Alderman Stairs remains an interesting piece of London’s architectural and historical legacy.“
I was really stunned with the amount of issues within this response.
Firstly, it reads like the answer someone would give on a school or university exam paper, where not enough revision has been done, and you have skim read at the last moment, and attempt to put together a reasonable answer using high level and half remembered facts (or was that just me!).
Secondly, the first paragraph on the location “Alderman Stairs is a historic set of stairs located in the City of London, connecting Upper Thames Street to Puddle Dock and the River Thames.” Alderman Stairs are nowhere near the location provided by ChatGPT, although I was concerned I may have missed something when writing about Puddle Dock, so went back and checked lots of maps, books etc. The following is Rocque’s 1746 and shows the street Puddle Dock (red arrow), Puddle Dock (yellow arrow) and where Puddle Dock accesses the Thames (blue arrow) – I could not find any reference to an Alderman Stairs anywhere near Puddle Dock:
Thirdly, the section on the origin of the name Alderman Stairs includes the following “The exact alderman behind the stairs’ naming is not fully documented, but it was common in the medieval period for prominent citizens to have public infrastructure named after them, especially in riverfront areas.“
There are some clear candidates for the source of the name, dating not from the medieval period, but from the late 17th / early 18th centuries.
As ChatGPT is a conversational tool, it offers the option for a follow-up question, so I asked “are you sure this is correct?”:
ChatGPT is certainly polite, as it apologised for the confusion, and confirmed that I was right to question the answer it had previously given “as there seems to be some inaccuracies in the historical details”.
In ChatGPT’s revised answer, the location of the stairs has now moved, and they are now “specifically close to the Cannon Street railway bridge”. I checked Rocque’s 1746 map, when obviously Cannon Street railway bridge was not there, the bridge started across the river where Steel Yard Wharf is shown in the centre of the following map:
There were no Alderman Stairs in the vicinity, or in later maps, including mid 20th century OS maps.
The rest of the answer is just as high-level and vague as the first answer. So my next question was “is there an information source for alderman stairs?”:
At first, my ego was somewhat deflated as my blog page on the stairs was not referenced, but on second thoughts I was rather pleased not to be associated with the quality of the ChatGPT response.
Even the list of sources raises some serious questions.
There are two books listed by Stephen Inwood, who has written a number of excellent books about London. I have the History of London, but I cannot find any reference to Inwood having ever written a book called “A history of the City’s Roads and Their Names”.
Likewise, the book “Old London’s Rivers and River Stairs” by E.S. McLachlan sounds a fascinating book, but again I can find no reference to such a book or author. I looked for these books at Amazon, Abebooks, the British Library and London Library, as well as a general Google search, but no luck.
I would be interested to know if these two books actually exist as I would like to have a copy.
In the Online Sources section, the Londonist website is listed. I tried a search at Londonist and there were no references to Alderman Stairs.
One of the tools provided by ChatGPT is image creation, and if you use Social Media you will find it flooded with images created by this type of AI tool.
I could not resist seeing what ChatGPT thought Alderman Stairs looked like, so I asked it to “Create an image of alderman stairs”, and this was the response:
A remarkable interpretation. I cannot argue with the “historical charm and tranquil atmosphere” of the real Alderman Stairs, and this is what they really look like:
By now, I was thinking that I was being a fit unfair with ChatGPT. Alderman Stairs is a rather obscure topic, and it would be reasonable for the tool not to be aware of such a place, although if that was the case, why does it just say that it does not know, rather than cobbling together a false answer. Part of intelligence has to be admitting when you do not know, rather than pretending that you do.
To see if I was being unfair to ChatGPT, I put the same question to other AI tools, the next was:
Microsoft Copilot
Microsoft Copilot has now got the correct location. It also provides a precis of the story of the stairs. The information is referenced, and at the end it provides the references, and links if the reader wants to know more.
The most used reference is to my blog, and a comparison of my blog post on the stairs with Copilot’s response shows where the information has come from, although it is a very high level summary.
I can see where this approach would be useful, as a quick way of finding information sources for a topic. Ask a question, and Copilot will provide a summary with a list of sources for follow-up.
I then put the same question to:
Google Gemini
The answer provided by Google Gemini is basically a summary of some of the key points from my blog post. At the end of each sentence, there is a button, and clicking provides the source of the information, as shown in the above example, and also in the example below:
Two points regarding Google Gemini’s response:
Firstly, the way these tools summarise gets rid of much of the context. In the above example there is a sentence on Irish immigrants seeking a new life in London.
In reality, these were very poor Irish people, probably close to starvation, and in the following screenshot from my blog post on Alderman Stairs, I have included the extract from the Illustrated London News which reported the landing at the stairs, and provides much more context:
My second concern is that when you click the dropdown box for the reference, Google Gemini states “Google Search found similar content, like this”. It is not “similar content” it is the original source information which Google has copied to use within its AI tool.
Getting rather depressed by now, I thought I would try one final AI tool. This tool is part of X (the old Twitter):
Grok
Again, some good information, but summarised and without the full context, for example, the Illustrated London News report I quoted in my blog post about the Irish Paupers is in the Grok response as “These stairs have been busy, with accounts mentioning up to 1200 people in one shipload, suggesting a high volume of traffic similar to that seen at Tower Bridge today” – and I have no idea why Tower Bridge is included as a comparison with Alderman Stairs.
Grok has a list of “Relevant Web Pages”:
Grok calls this listing “Relevant Web Pages”, where in reality, these are pages and websites where Grok has sourced the information to compile the response, including my blog.
This very quick look, using a single question, raises a number of questions:
Copyright
AI systems are trained on data which is electronically available and much is sourced by searching the Internet. AI tools then use the information found to build a response to a question and provide this as the AI’s answer to the question.
AI systems are therefore using the work of other people, authors, and organisations, and where the source is given, with Google it is quoted as “similar content”.
Microsoft Copilot was the best of those tested in providing links to the sources used to build a summary response.
At the moment, if you search for Alderman Stairs on search engines such as Google, you will get a link to my post. As AI tools improve, they will end up showing a comprehensive answer, thereby reducing the incentive for people to find the original or alternative sources, so AI tools will use data from other websites whilst at the same time reducing the visibility of the sites which provided the source information.
Currently, the UK Government are consulting “on proposals to give creative industries and AI developers clarity over copyright laws”.
The key points from this consultation are:
It will be interesting to see how this develops, however there is not a good track record in the protection of data on the Internet, particularly where the big tech companies are concerned.
Historical Accuracy
The use of these tools means that there is far more risk that information becomes distorted, provided out of context, or is just plain wrong.
It is also easy to see how the response from AI tools can be manipulated as they are basically building a response from the information they have found at other sources. If those sources are using false information, AI tools may probably just repeat this.
Much of the ChatGPT response was just so wrong, and users would have far more confidence in the output of these tools that when an AI tool does not have the information, it just answers with an “I do not know”.
I have a very amateur interest in London’s history, do this for my own interest, and fortunately others find my content interesting as well, however for anyone who writes professionally, depends on writing for their income, carries out academic research etc. I would be concerned about where this is going.
ChatGPT did however redeem itself with a final test. As I was finishing this post, I thought it would be interesting to see what ChatGPT knew of my blog, and this was the answer:
I really like ChatGPT’s summary, and it is a far better summary of my blog than I think I could write – so you can see where Artificial Intelligence can be useful, and I might use the above text as my new “About” page.
Artificial Intelligence is not going away, and if you would like to try out these AI tools:
- Click here for ChatGPT
- Click here for Microsoft Copilot (scroll down to the bottom of the Ads)
- Click here for Google Gemini
- Grok can be found if you have an X account
Whatever the future of AI, I can promise you that all my blog posts will always be written by a human, using old books, maps, library and archive research, newspaper archives, photos and images, visits to site etc. and will come with the poor grammar, punctuation and occasional typos, that comes with keeping up the amateur production of a weekly post, and as ever, I really appreciate any corrections.