Chat with RTX, Nvidia’s ChatGPT that we can run without Internet

Today, I would like to discuss a new product that will certainly attract the attention of technology and artificial intelligence enthusiasts, particularly those who are interested in hardware and software and want to explore the capabilities of AI on their own machines. This product is called Chat with RTX, and it has been launched by Nvidia. It promises to transform the way we interact with generative AI models directly from our PCs.

Nvidia is a leading company in the field of graphics solutions. The company has also made significant strides in the field of artificial intelligence. To promote its latest graphics cards, the GeForce RTX 30 and 40 Series, Nvidia has introduced Chat with RTX. This feature enables owners of these cards to run an AI-powered chatbot offline on a Windows PC.

This tool has an interesting feature that enables users to personalize a GenAI model in the style of OpenAI’s ChatGPT. It can connect to documents, files, and notes to provide specific answers to queries. For instance, you can easily ask “What was the name of the restaurant that my partner recommended when we were in Barcelona?” and Chat with RTX will scan the relevant local files to give you a contextual response. Such customization offers a range of possibilities for personal and professional productivity, allowing us to access information swiftly and efficiently without having to manually search through notes or saved content.

Here you have it working:

The default tool employs an open-source model from AI startup Mistral, and it also works with other text-based models, such as Meta’s Llama 2. However, it’s important to note that downloading all the required files may require a significant amount of storage, ranging from 50GB to 100GB, depending on the model you choose. This is a crucial consideration for anyone interested in using this tool, as it requires an investment not only in the graphics card but also in storage solutions.

Although Chat with RTX has its advantages, it also has its limitations. For instance, the tool is not capable of storing the context of previous inquiries, which can restrict the flow of conversations. Furthermore, the accuracy of the responses can be influenced by different factors, such as the phrasing of the question, the effectiveness of the chosen model, and the extent of the fine-tuning data set.

Nvidia’s initiative is notable for its ability to simplify the process of running AI models locally. This approach is becoming increasingly popular, as the World Economic Forum predicts a significant rise in affordable devices that can run GenAI models offline. This approach offers clear benefits, such as increased privacy since the processed data never leaves the device, reduced latency, and better cost efficiency when compared to cloud-hosted models.

However, it’s important to consider the potential risks associated with the democratization of tools for running and training models. While this technology has the potential to revolutionize productivity and beyond, it could also open the door to malicious actors. At Mediaboooster, we emphasize the importance of using technology responsibly and being aware of potential risks.

We will continue to monitor the evolution of this technology and its impact on the field of productivity and beyond.