AI Topic-X2: Programming Experience with GPT-4
AI Topic X2: The Experience of Programming with GPT-4
GPT-4 is an incredibly powerful AI, but the vast majority of people simply don’t let it be fully utilized. Treating ChatGPT as a chatbot for casual fun is a disservice to your own time and money as well as to GPT. if you had a robotic cat in your house, would you just play brain teasers with it?
GPT is not a chatbot, it’s a “generative” AI. generative in the sense that it creates content. How far it can go is not up to it, but up to you. Being able to use GPT to flying colors will probably be one of the few most important skills you’ll ever have in your life. This AI topic of ours will be a comprehensive look at the skills of each GPT application. I am also learning and practicing as I go along ……
In this talk we explore one of the most powerful GPT skills, programming. This is OpenAI’s unique skill, now several big companies have launched their own big language models, some have more parameters than GPT, but those who can program are probably the only ones who can program GPT.
The little project I worked on was to create an idiomatic connect-the-dots game. The programming language I chose was Python, but I hadn’t written a program in Python in a long time, so this time I simply pretended not to know anything at all and started from 0. The real programmer this time was not me, but the GPT-4 model under the ChatGPT interface. The whole process we both communicated in Chinese.
✵
We first needed to verify that GPT did have what I wanted. I started by having ChatGPT list 10 random idioms and provide their explanations. It did a great job of it -
Then I started making direct demands by saying “Write a Python program that separates these idioms and their explanations, lists them separately, breaks up the order, and lists them in two columns on an A4 sheet of paper to form a connect-the-dots question”.
You see I’m being rather general. I didn’t tell it how to do it, I didn’t design a detailed algorithm. I just mentioned a requirement. It turns out that this small requirement GPT is perfectly capable of handling.
It understood my request immediately, but instead of starting programming straight away, it first reminded me to install a ‘program library’ because this library is not on every computer and it would be used for this programming:
Then it clatters out the whole program. I’ll show you a screenshot where it marks up each section of the program that it implements with comments to make it easier for me to understand -
Of course, being zero-based, I didn’t want to understand. The whole program is only 70 lines or so, very simple.
I ran the program and immediately got an error. I understood the error message to be about a font error; GPT uses English fonts and can’t display Chinese, which it obviously hadn’t thought of beforehand. I told it about the problem, and it corrected it immediately, saying “I’m sorry for the inconvenience”–
It also told me where to find Chinese fonts. But after I run it I get the error again. This time I didn’t even look at the error message, I just copied it to it, and it realized that I needed to mount a separate library to add Chinese fonts, and told me what to do, and rewrote the whole program –
There is a font in the middle of the path still need to change, and then the program ran successfully. Generated a PDF file, open is a set of idioms to explain the connecting questions –
Maybe you can do it for fun, but that’s not quite what I want. What I want is to generate a new set of questions every time I run the program, to the point where I can do it myself for fun. I can’t use ChatGPT to generate idioms before copying them into the program every time, I have to automate that to the point where I can get a new set of questions with a single command.
To do this, we have to call OpenAI’s API, which is what it’s all about, and allows your machine to talk directly to GPT remotely and return results automatically.
✵
This is the first time I’ve used the API, and I have zero experience with it. But that’s okay, as soon as I talked to it, ChatGPT was kind enough to tell me what I had to do first to use the API-
I followed the steps it gave me exactly and it clattered the whole program out. And it remembered that we need Chinese fonts …… Here’s a small detail is that GPT-4 didn’t know that OpenAI had opened up the API for GPT-3.5 and that the traffic price was ten times cheaper, it gave the old model at first, and I changed it to the new one by myself (model=”gpt-3.5-turbo” ).
As soon as I ran it, I got an error again, I copied the error message to it, and it immediately realized what the problem was and made the changes -
Then it says, “Now the program should work fine. Please note that there may be additional costs associated with using GPT-3.5. Make sure you are aware of the costs before using it.”
How thoughtful of me to waste my money. After a few more of these back and forth error corrections, the program finally ran smoothly, indicating that the API call was successful, but the output looked like this -
Not only is the formatting incorrect, but the idioms and explanations are not separated and out of order. I had no way to show ChatGPT the diagram, so I had to describe the problem in words, and it understood it as soon as I heard it!
It modified the program, only to make several more errors as soon as it ran it. At this point it used a common programmer’s approach to debugging the program: it asked me to insert print statements in several places in the middle of the program and look at the intermediate results, as a way to find out where the problem actually lies -
I ran the program, told it the results, and then it looked for the problem–
After a few tossups like that, we – me and it – both realized that it was because the GPT-3.5 API output wasn’t in a standardized format, and that you had to come up with your own way of distinguishing where the idioms were, and where the idioms were interpreted. I didn’t say anything, ChatGPT figured it out on its own - look for the colons in it! There’s a colon between the idiom and the explanation, which should be distinguished by a colon.
That hurdle passed, and the program ran successfully. But the output was less than ideal –
After some more tossing and turning, the program suddenly reported another error. This time the error was very interesting. It turns out that GPT-3.5, as a language model, sometimes changes its tone of voice when talking to you, for example, it starts with a polite sentence: “OK, here are 10 Chinese idioms or fixed phrases and their explanations:” - it doesn’t matter that it is polite. ChatGPT’s previous formatting with colons doesn’t work well.
But ChatGPT immediately came up with another idea - the
which equates to a smarter way of finding idioms and explanations. This worked, but then ChatGPT simply modified the prompt I originally wrote for GPT 3.5 so that it had to follow the standard format every time–
I’m ashamed that I didn’t think of that at first.
Finally the program worked, and the output looked like this -
Each run is a new set of questions, where the order of the idioms and explanations is scrambled, requiring you to connect the dots –
The layout is actually not pretty enough, but I’m satisfied. You can also turn the explanations into English and practice your English with this program ……
It took more than three hours in total.
✵
It was an amazing programming experience for me. There were only two times I took the initiative to make adjustments to the program, one was that the Chinese font path given by ChatGPT was incorrect, and the other was that there was a small problem with the order of the definition of one of the functions at a later stage, which might be due to repeated modifications, and I changed it by hand when I saw it. In fact, my main job is to strictly follow the requirements of ChatGPT to modify the program, run the program and report the error information back to it.
It’s perfectly capable of doing these things on its own. I understand that OpenAI prohibits the GPT from programming directly for safety reasons: you really don’t know what it can program.
Of course I’m not contributing nothing at all. My main role is to mention requirements. The idea of the program is mine, the “grand design” of the API is mine, and the decision to accept or reject the final result is mine. …… Even the layout of the final presentation is done by GPT.
This is “natural language programming”. I’ve never used Python to call up APIs or generate forms before, I don’t know - and I don’t need to know - what the specific programming statements are, I just have to look at them with the help of annotations to see what they mean. I didn’t even have to design - or even think about - the algorithms for manipulating idioms and interpreted strings in the middle. Think of how much time and effort that saves.
GPT-4 didn’t generate the perfect program all at once, but that merely shows that it’s not a god! It proved itself to be a very good programmer. There is not a single low-level error in the program. All of the errors were either related to the specific runtime environment or to the results returned by the GPT-3.5 API, and were problems that you had to try to find out.GPT-4 didn’t waste a single step in the entire process. Every error message returned by the machine, every phenomenon I mentioned, it understood correctly and got to the heart of the matter, and immediately provided a solution.
I was most amazed by the initiative and creativity shown by GPT-4. It took the initiative to let me add a few intermediate results to my program to help me debug it. In the face of the irregular output of GPT-3.5, it first creatively thought of using “colons” to determine the distinction between idioms and explanations, then invented a more adaptable search method, and finally modified the hints I wrote directly.
Could a human programmer have done a better job working with me?
In fact, during the programming process, I subconsciously, have treated it like a human being. Sometimes I’m instinctively embarrassed to give it the error message again–I almost want to tell it, “Why don’t we take a break and work on it later–I have to convince myself by sheer force of will that it’s just an AI, that it’s not afraid of trouble and not afraid of getting tired! before I could continue.
Whenever I get an error message, ChatGPT always says sorry to me first. How does it know that it is actually me who feels sorry …… is really bothering people.
I don’t know what your definition of AGI is, it looks like AGI to me.
✵
Will GPT replace the work of human programmers? At least not at the moment. Programmers should have a better idea of what a project manager needs than an AI, and for bigger programs, it’s probably still necessary to have a programmer designing the big picture. But perhaps the more fundamental reason is that GPT is not allowed to program independently: a human must compile, run the program and report errors for him to do so.
But the GPT would greatly improve the programmer’s programming efficiency. You don’t need to memorize specific program statements or design small algorithms anymore; GPT is your most faithful and competent assistant. And you don’t even have to program alone from now on, you gain a sense of companionship.
But GPT’s biggest contribution is still that it allows people like me, who don’t usually program, to program. I can’t spend five hours a day programming, but you’d have to say three hours a week, and that’s very pleasant for me. There are countless people in the world who have ideas but no time, and now with GPT, all these people can work on their ‘secret projects’ immediately.GPT is empowering, it liberates us.
Just a few short days after the release of GPT-4, so many people on Twitter showed their programming projects using GPT-4. Some people made applets that use AI to process documents and speech, some made browser plug-ins, some made mobile apps, and a few made desktop video games. Many of these were people who don’t normally program.
The fact is that GPT has not only made programming easier, it’s made it more worthwhile. Because you can now call AI in a program! Having AI in a program is absolutely picture perfect, it comes alive and it can do all sorts of amazing things. Who would have thought before that a person could write a program at home that would automatically come up with idiomatic questions? I have several other interesting ideas, and I’m going to find the time to make them happen.
✵
I remember the day GPT-4 was first released and everyone on Twitter was incredibly excited to try it out in various ways. At night I watched as a buddy said: guys stop playing and go to bed first, you’ll wake up tomorrow and GPT-4 will still be there.
This programming experience really gave me that feeling. I don’t want to leave it for a minute, I’m afraid it’s gone. I’m really afraid of waking up tomorrow and realizing that it was all a dream ……
Getting to the point
GPT will greatly improve the programmer’s programming efficiency, it not only makes programming easier, but also makes it more worthwhile.