Alright I guess its about time I write something about AI. Yeah I’m pretty late to the party on this one, but my take on ChatGPT isn’t something I’ve heard talked about much on the inter-webs. One thing I must disclose, although I am an absolutely briliant engineer, AI is a bit outside of my expertise, but when has that ever stopped people from yapping their mouths on the internet?
I am going to boil down AI language learning models into a commically simplistic form. Fair warning for any real AI engineers reading this, I’m probably about to make you cringe. ChatGPT is essentially an absurdly complicated math formula. It’s input is a number, and its output is a number. Now some of you might be thinking “this idiot has no idea what he’s talking about, ChatGPT takes in sentences and outputs sentences, not numbers”. Well in a way, both are true (you should check out that Substack by the way). When you type “What is the meaning of life?” into ChatGPT the actual machine learning model gets this : 01010111 01101000 01100001 01110100 00100000 01101001 01110011 00100000 01110100 01101000 01100101 00100000 01101101 01100101 01100001 01101110 01101001 01101110 01100111 00100000 01101111 01100110 00100000 01101100 01101001 01100110 01100101 00111111. Now, I have made an assumption that ChatGPT converts strings into binary sequences using the ASCII standard. I don’t know if OpenAI programmed ChatGPT to use that standard but in any case my point is still true. ChatGPT does not understand words or letters, it understands binary sequences, which are essentially numbers. This is also the case for the language model’s output. It spits out a binary sequence that is then converted into letters that us humans can understand.
Think about it like this, I hand you a seemingly magical black box. The left side of the box has millions of switches, each switch has only two positions, up or down (1 or 0). You have the ability to flip these switches into any combination of up and down you’d like. This is your input to the black box. Now, the top side of the box has millions of knobs, think of the volume knob on a car stereo, and the right side of the box also has millions of switches, however these switches are not the input, these switches are the output of this black box. In very very simple terms this is the architecture of machine learnign models. Let me explain what the knobs are for. After you have set the input switches, you can start messing with the knobs. When you turn a few knobs, some of the switches on the output side flip positions. The programmers at OpenAI have the ability to tune these knobs to get a desired output from a specific input. Well… they kind of have this ability. Tuning these “knobs” manually on something the size of ChatGPT would take trillions of years. So, the model is actually tuned during a training session. If I explained the training of machine learning models in this post it would get rather lengthy, so let me know in the comments if you would like to see a future post about this. Or, feel free to tell me I’m making no sense at all and I should just shut up! So this is what the great ChatGPT is, a black box with a binary string input (the switches on the left side), a complicated math formula applied to the binary input (the value that the knobs are set to), and a binary string output that is the result of the math formula applied to the input (the switches on the right side).
So what’s the big deal then? Here’s my hot take: I think people are over reacting when they say we should be scared or cautious in the development of these models. The main grevience I have heard is that people are gonna believe everything ChatGPT says and we’ll live in a society of phonies. Well, aren’t we already there? People already believe everything they see on the front page of Google, Facebook, Twitter, Reddit, (insert other part of the internet devoid of benevolence here) etc,. I can’t really see how ChatGPT’s output could get much worse than what is posted daily on Twitter and 4Chan. “Well, what if one day it figures out how to write code to make itself better and it grows into something uncontrollable?” I think we’re pretty far away from this being a possiblity with todays machine learning architectures. I can tackle this more in my future post about the training process, but essentially I don’t think this can happen because presently it takes some amount of human input to train and improve these models. It cannot happen autonomously. With all this being said, I absolutely belive that AI will revolutionize almost all industries. How and why? Well there’s a surplus of articles and Substack posts you can read about this, but again let me know if you’d like to see my take on this.
I’ve rambled on for too long now, I hope you learned something and I’ll let you get back to the rest of your day!
I agree, everyone believing that’s not idiot is equally idiot