Researchers from Stanford University’s Artificial Intelligence (AI) team were able to develop their ChatGPT chatbot, Alpaca, in less than two months. However, they decided to terminate it shortly after release, citing the high costs of hosting and inadequate content filters for the model’s behavior. The announcement of termination was made just a week after the release, as reported by Stanford Daily.
Despite being developed for less than $600, the source code for the ChatGPT model is publicly available. According to the researchers, the performance of their chatbot model was similar to OpenAI’s ChatGPT 3.5. They emphasized that the chatbot is solely for academic research and not intended for general use in the near future.
Alpaca researcher Tatsunori Hashimoto, from Stanford’s Computer Science Department, stated that the team believed that the “interesting work is in developing methods on top of Alpaca” and that the dataset itself is just a combination of known ideas. Therefore, they do not plan on creating more datasets of the same kind or scaling up the model.
Alpaca was developed on Meta AI’s LLaMA 7B model and generated training data with the self-instruct method. Adjunct Professor Douwe Kiela, who also worked as an AI researcher at Facebook, noted that “as soon as the LLaMA model came out, the race was on.” He also commented on Alpaca’s popularity, saying that “it’s a really, really cool, simple idea, and they executed really well.”
Hashimoto explained that “the LLaMA base model is trained to predict the next word on internet data, and instruction-finetuning modifies the model to prefer completions that follow instructions over those that do not.” The source code for Alpaca is available on GitHub, and it has been viewed 17,500 times. Over 2,400 people have used the code to develop their own model.
However, Hashimoto also mentioned that “much of the observed performance of Alpaca comes from LLaMA, and so the base language model is still a key bottleneck.” As the use of AI systems increases, scientists and experts continue to debate the publishing of source code, data used by companies, methods to train AI models, and overall transparency of the technology.
Kiela believes that “one of the safest ways to move forward with this technology is to make sure that it is not in too few hands.” He added that “we need to have places like Stanford, doing cutting-edge research on these large language models in the open. So I thought it was very encouraging that Stanford is still actually one of the big players in this large language model space.”
Despite the termination of Alpaca, the project has generated a lot of interest in the AI community, particularly due to its rapid development timeline and relatively low cost. Many researchers have been experimenting with the source code and using it to train their own models, demonstrating the value of open-source sharing in the field of AI research.
The use of large language models like GPT-3 has become increasingly popular in recent years, with companies using them for a variety of applications, including chatbots, language translation, and content creation. However, there has been growing concern over the potential misuse of these models, particularly in generating fake news and spreading misinformation.
As such, the debate over transparency and ethical use of AI continues to be an important topic of discussion within the scientific community. The release of Alpaca’s source code serves as a reminder of the importance of open sharing in advancing AI research and ensuring that the technology is developed and used in an ethical and responsible manner.