CodeGPT can definitely produce code that runs, but can it reliably produce good, high-quality code?
If you're looking for an AI-powered assistant to help you write code, chances are you've encountered CodeGPT. It's one of many AI-powered tools you can use to assist you when programming. But can CodeGPT actually write code?
What Is CodeGPT?
CodeGPT is a dedicated extension that uses different artificial intelligence (AI) models to help programmers write and fix code. It includes various features geared towards faster and easier programming, including auto-completion, code explanation, refactoring, documentation, unit testing, error-checking, and bug-fixing. It also has a ChatGPT-like interface you can access from your code editor.
The extension is particularly powerful because it lets you connect to various mainstream large language models (LLMs) from different providers like OpenAI and Google. On top of that, CodeGPT lets you create your own AI agents that you can use in your projects or share with others.
While there are several code editors for Linux, macOS, and Windows, CodeGPT is only available in two. Today, you can either install and use CodeGPT in VS Code or in Cursor, which is a fork of VS Code.
The extension is particularly powerful because it lets you connect to various mainstream large language models (LLMs) from different providers like OpenAI and Google. On top of that, CodeGPT lets you create your own AI agents that you can use in your projects or share with others.
While there are several code editors for Linux, macOS, and Windows, CodeGPT is only available in two. Today, you can either install and use CodeGPT in VS Code or in Cursor, which is a fork of VS Code.
How Much Does CodeGPT Cost?
You can download and install CodeGPT for free, but using the extension comes at a cost. CodeGPT offers various subscription packages that you can pay for if you need unlimited access.
CodeGPT Plus has three subscription plans. The Basic plan goes for $9.99/month, followed by Standard ($19.99/month) and Gold ($49.99/month). There's a free trial period available, which you can use to evaluate the extension. Each package gives you access to specific AI agents with unlimited interactions.
Bear in mind that you'll need to pay for unlimited usage of third-party AI models like OpenAI's GPT-4 model, for example. Although you can access Open AI's GPT-4 model for free, you cannot do so from within VS Code or Cursor.
CodeGPT Plus has three subscription plans. The Basic plan goes for $9.99/month, followed by Standard ($19.99/month) and Gold ($49.99/month). There's a free trial period available, which you can use to evaluate the extension. Each package gives you access to specific AI agents with unlimited interactions.
Bear in mind that you'll need to pay for unlimited usage of third-party AI models like OpenAI's GPT-4 model, for example. Although you can access Open AI's GPT-4 model for free, you cannot do so from within VS Code or Cursor.
Can CodeGPT Really Write Code?
To evaluate this, we'll use the extension to help us generate two different code samples in Python.
First, we ask the model to generate a stack data structure in Python with all the relevant methods. We used this prompt:
First, we ask the model to generate a stack data structure in Python with all the relevant methods. We used this prompt:
Create a stack in Python and include all the necessary methods
The model generated the following response:
class Stack:
def __init__(self):
self.stack = []
def push(self, item):
self.stack.append(item)
def pop(self):
if self.is_empty():
raise IndexError("Pop from empty stack")
return self.stack.pop()
def peek(self):
if self.is_empty():
raise IndexError("Peek from empty stack")
return self.stack[-1]
def is_empty(self):
return len(self.stack) == 0
def size(self):
return len(self.stack)
This sample shows that CodeGPT is clever enough to create the stack using a class and include the necessary checks to handle specific errors that may arise. For instance, the code throws informative errors if you try to peek or pop an empty stack.
In a second challenge, we asked CodeGPT to save a response formatted in JSON from the JSONPlaceholder API to a CSV file. For context, the response from the JSONPlaceholder API looks like this:
In a second challenge, we asked CodeGPT to save a response formatted in JSON from the JSONPlaceholder API to a CSV file. For context, the response from the JSONPlaceholder API looks like this:
{
'userId': 1,
'id': 1,
'title': 'sunt aut facere repellat provident occaecati excepturi optio reprehenderit',
'body': 'quia et suscipit\nsuscipit recusandae consequuntur expedita et cum\nreprehenderit molestiae ut ut quas totam\nnostrum rerum est autem sunt rem eveniet architecto'
}
We asked CodeGPT the following:
I've made a request to an external API at https://jsonplaceholder.typicode.com/posts/1 and I'm getting a response back. I've formatted the response as json, how can I save it in a csv file?
The code that CodeGPT responded with was:
import csv
import requests
# Make a GET request to the API
response = requests.get('https://jsonplaceholder.typicode.com/posts/1')
# Convert the response to JSON format
data = response.json()
# Since we're receiving a dictionary, we need to turn it into a list of one dictionary
data = [data]
# Open (or create) a CSV file to write the data
with open('output.csv', 'w', newline='') as output_file:
# Create a CSV writer object
writer = csv.DictWriter(output_file, fieldnames=data[0].keys())
# Write the CSV headers
writer.writeheader()
# Write the data to the CSV file
writer.writerows(data)
Generally, this response is good; it doesn't need any tweaking to work. CodeGPT also added comments explaining the purpose of the various lines of code, which is useful if you're doing something for the first time.
The only issue here is the response doesn't specify an encoding type while calling open(). While a default will automatically be included, specifying an encoding type is recommended and considered best practice when writing to files in Python. This might save you from incorrect interpretation, leading to jumbled text.
So: CodeGPT can write code, but you should be careful because it won't always give you the best output. What you get might be error-prone and might not follow the best practices. Whenever you use CodeGPT or any other AI-powered tool to write code, strive to understand what the code does first. Don't just copy and paste.