Like blockchain, there’s a lot of hype around AI. But unlike most of blockchain, it’s here to stay, driving significant disruptions in software development and redefining knowledge work as we know it. Let’s delve into a simple example, then explore the current applications of AI in development pipelines at Mycos.

AI in Action: Code Interpreter and Unit Tests

Code Interpreter is OpenAI’s newest rollout, and many are calling it GPT-4.5. Available only to plus users ($20 / month), it adds the ability to execute and manipulate code and data.

We upload a fairly simple Python application that reads some csv data and then does a calculation:

We then instruct it to query for unit tests and write some:

And finally, our revised application, with unit tests, is made available for download:

The generated tests:

import unittest
import csv
from pathlib import Path


# Define necessary functions
def read_sales_data(filename):
    with open(filename, 'r') as f:
        reader = csv.reader(f)
        return list(reader)


def calculate_total_sales(data):
    total_sales = 0
    for row in data[1:]:
        try:
            revenue = float(row[-1])
            total_sales += revenue
        except ValueError:
            raise ValueError("Invalid data encountered: {}".format(row))
    return total_sales


# Define test class
class TestSalesData(unittest.TestCase):
    
    # Set up test data files
    @classmethod
    def setUpClass(cls):
        cls.valid_data_file = "valid_data.csv"
        cls.invalid_data_file = "invalid_data.csv"
        
        valid_data = [
            ["Year", "Month", "Customer Age", "Age Group", "Customer Gender", "Country", "State", 
             "Product Category", "Sub Category", "Product", "Order Quantity", "Unit Cost", 
             "Unit Price", "Profit", "Cost", "Revenue"],
            ["2023", "July", "30", "30-35", "M", "USA", "California", "Accessories", "Bottles", 
             "Water Bottle", "1", "5.09", "20.98", "7.98", "5.09", "20.98"]
        ]
        
        invalid_data = valid_data + [["2023", "July", "30", "30-35", "M", "USA", "California", 
                                      "Accessories", "Bottles", "Water Bottle", "1", "5.09", 
                                      "20.98", "7.98", "5.09", "invalid"]]
        
        # Write test data to CSV files
        with open(cls.valid_data_file, 'w', newline='') as f:
            writer = csv.writer(f)
            writer.writerows(valid_data)
            
        with open(cls.invalid_data_file, 'w', newline='') as f:
            writer = csv.writer(f)
            writer.writerows(invalid_data)
    
    # Remove test data files
    @classmethod
    def tearDownClass(cls):
        Path(cls.valid_data_file).unlink()
        Path(cls.invalid_data_file).unlink()


    def test_read_sales_data_valid(self):
        data = read_sales_data(self.valid_data_file)
        self.assertIsInstance(data, list)
        self.assertEqual(len(data), 2)
        self.assertEqual(len(data[0]), 16)
        self.assertEqual(data[0][0], "Year")
        self.assertEqual(data[1][0], "2023")


    def test_calculate_total_sales_valid(self):
        data = read_sales_data(self.valid_data_file)
        total_sales = calculate_total_sales(data)
        self.assertEqual(total_sales, 20.98)


    def test_calculate_total_sales_invalid(self):
        data = read_sales_data(self.invalid_data_file)
        with self.assertRaises(ValueError):
            calculate_total_sales(data)


# Run the tests
if __name__ == '__main__':
    unittest.main()

Other things we found in the process:

• Gently pointed out we could do validation at the time of the CSV load (code critique).
• Gave us a very nice description of what the app does.
• Read and (attempted) to graph the contents of the sales data.
• Ran the unit tests it wrote to validate everything.

Where We See Immediate Benefits

Feedback from our engineers has highlighted several areas where AI integration offers immediate benefits.

1. Requirements Engineering and Task Breakdown

AI seems tailor-made for breaking down complex projects into specific tasks, particularly when utilizing language models. By using natural language processing (NLP), AI can streamline the requirements engineering process, understand project specifications, and automatically generate user stories, use cases, and system requirements. This not only saves time but also minimizes potential misunderstandings by providing more accurate and detailed requirements.

2. Language Mentoring and Code Translation

With AI tools like ChatGPT and Bard, developers can access quick and generally accurate translations between programming languages. These AI-powered tools facilitate learning new languages and maintaining codebases written in multiple languages, enabling developers to focus more on reviewing code translations, problem-solving, and higher-level design. This approach not only boosts efficiency but also reduces time spent on syntax errors and other low-level issues.

3. Code Generation

While AI might not excel at analyzing or producing large complex code bases, it’s adept at generating discrete functions for review. This capability is revolutionizing how developers write code. Now, instead of starting from scratch, developers can use AI to generate code snippets for specific functions, accelerating the coding process and reducing potential errors. Furthermore, AI can suggest improvements to existing code snippets, enhancing both performance and maintainability.

4. Testing

AI excels at generating test datasets and identifying edge-case scenarios for unit testing. It can automatically generate a range of test scenarios that may not be readily apparent to human testers. This ensures that software is robust against unexpected inputs or conditions, leading to more thorough testing and, ultimately, higher quality software.

5. Limits and Challenges

For domain-specific tasks, AI models are only as good as the data they were trained on. Technology innovations happen quickly, and the context provided by large language models will lag accordingly. Overreliance on AI in areas like cybersecurity, where new attack vectors emerge almost daily, would be naïve.

Final Thoughts

The value in the current AI tooling has led us to purchase OpenAI API subscriptions for our engineers and provide training on how to leverage these and other tools in design, development, and testing workflows. We anticipate that AI-driven development tooling will mature rapidly.

As AI continues to reshape the landscape of software development, it’s crucial for professional services organizations to adapt and evolve. Just as professional translators today primarily fine-tune AI-generated translations, the value-added software engineers of the future will largely supervise generated code.

However, existing codebases are now ripe for bullet-proofing with additional tests and validation.

At Mycos, we not only welcome this evolution but also look forward to driving it, both through workflow and productivity enhancements and by incorporating AI in client products. Stay tuned for more exciting developments.

If you need help getting your development team up to speed on the application and use of AI, contact us today.