05/31/23 | EverOps
AI technologies are moving fast! It is hard to keep up and know where things are. The intent of this post is to reflect on the current state of AI technologies and a brief analysis of their use and potential benefits.
The entry barrier is not very clear, most will try and attempt to search on the internet for sources and materials related to AI and discover there are many paths taking you to different destinations. It is after we took on that effort ourselves, that we decided to share our opinions in order to help others get started in this arena.
Currently, there are many opinions about AI and how to use this new technology. During our research, it became very clear that there are opposing views with regard to the safety and dangers of this emerging technology without any guardrails or guidelines. Currently, the main players for these technologies are in talks with governments to try and lay down the foundation towards regulations and safety thresholds for determining up to what point AI can be both trained and used.
Most Technologies tend to evolve, they tend to mature and those successful tend to become part of our day-to-day lives. Chances are, AI is here to stay. There are indeed some dangers, however, it is up to us to make good use of the technology and its practical applications. Hence in this particular post we will focus on the positive and the potentially beneficial opportunities AI provides.
At EverOps, we made resources available to investigate and provide the basis of a company opinion as well as suggestions for use cases and how to handle the limitations/dangers of these new technologies.
There are a few different AI Technologies that, even though in the same arena, perform different functions and have different purposes. For instance, general generative chatbots and AI paired code development assistants.
In the case of generative chatbots like chat OpenAI GPT (https://openai.com/product/chatgpt), and Google Bard (https://bard.google.com/) these are absolutely great tools to help assist, augment, and empower professionals to make them more efficient, more effective, and able to focus on the more mentally challenging tasks which require experience, know-how, and human criteria for decisions. The entry point to being able to successfully use these tools is learning how to communicate with them. And with that a new field was born, Prompt Engineering (LINK TO ARTICLE), the science behind natural language processing for guiding AI towards more complete and accurate answers. Which also applies to other related cases like AI paired programming that offers autocomplete-style suggestions as you code.
A simple example for general use of ChatGPT was the case where we got instructions from the web on how to install an Open Source application into our kubernetes cluster, but the deploy system we were using required a different format for the configuration file. During a work session some of us started translating this configuration into its final form, however, one of us asked ChatGPT to translate it, the result was almost instantaneous… and in this case, correct. The lesson here for us is twofold, one is that some of us are used to solving problems ourselves, that’s how we’ve been doing it forever. Others have realized that this is a tool that they can use to produce things faster, not necessarily better but faster. Then analyzing and improving the results to come up with a better solution… and that combination creates a faster and better outcome than if not using these tools.
We also did an experiment to see if AI paired coding was ready to be used and unsurprisingly we realized that we needed some training to understand how to use it properly. Prompt Engineering (LINK TO ARTICLE) also plays a large role here, and the understanding that the generated code is a suggestion, details need to be filled, and logic adjusted when complexity increases. However, we believe we did indeed become more efficient at writing code, but having to do so at a higher level and then reviewing the details of the AI generated code. At times this exercise was simple and straightforward, but at times the mistakes, or better yet, insufficiencies made by the AI (maybe due to our prompts) were so subtle that even during a comprehensive review they could have been easily missed. In its current form, AI is a good assistant but still far from being able to write comprehensive code for complex applications.
Lastly, we ran our recruiting interview questions by ChatGPT and we observed interesting results. For the question set provided by one of our more introverted, no nonsense, highly organized engineers the AI Candidate achieved a 95% score, higher than any human contender. On the other hand, the question set provided by an outgoing, more social, verbose engineer, rendered a mediocre score, with several wrong or incomplete answers. In the end, it goes back to how we communicate with the AI and adjust our natural language to interface with such systems. This however, seems to be evolving, and a mature AI will be able to understand with less, or maybe even combine different inputs like voice intonation, facial expressions, body temperature, etc…
That being said we also realize that there are a few limitations to the current state of AI with regards to these technologies. The main one is the training material. We ran into an issue where a newer version of a library existed and the AI – unaware of it, was suggesting code incompatible with the latest version we had been using. Along the same lines, is the fact that we cannot use company or sensitive data to train these providers because there is no assurance that the data will be siloed and scoped for our use only. Therefore, it makes it almost impossible to use the current AI offerings to analyze our infrastructure and provide guidance towards optimizations and security risks. We can only use what exists, from a period of time where each particular tool was trained, currently one or two years ago.
In summary, our first impression of the current state of AI is very positive. These tools can be used right now to increase our efficiency and focus on tasks better suited for humans. Chatbots are a great way to get information in easy-to-consume bits, and AI paired coding can produce clean and optimized code in most situations. And even though it does have some shortcomings and limitations, we understand that this technology is changing and evolving and with it, we should embrace it and evolve as well.