[AINews] MetaVoice & RIP Bard • ButtondownTwitterTwitter

TheBloke Discord Summary

• AI Training Conversations Heat Up: Discussions involved AI models like Mixtral, Nous Mixtral DPO, and Miqu 70B, comparing them with OpenAI's GPT models on efficiency and capability. Debates flared on the Reddit /r/LocalLLaMA subreddit's moderation with shared links to GitHub, Hugging Face, and YouTube videos discussing AI advancements and issues within the community.

• Antiquity Meets Modernity in AI Nomenclature: In the #characters-roleplay-stories channel, Thespis 0.8 sparked a debate about its Greek tragedy origins, turning the conversation towards the use of mythology for AI context. The practice of lorebooks in roleplay was discussed as a tool for prompt engineering, and DPO (Direct Preference Optimization) was mentioned with example wandb links provided.

• Removing Safety Features for AI's Full Potential: Users shared insights on LoRA fine-tuning to remove safety guardrails from models like Llama2 70B instruct, citing a LessWrong post. Discussions also suggested combining datasets could enhance model control during fine-tuning.

• Finding Tech Limits in AI Development: Query on the possibility of offloading transformer layers to a GPU as observed in llama.cpp, led to the conclusion that the Transformers library doesn't support layer splitting between CPU and GPU. Interest was shown in Meta's Sphere project, considered for its potential of incorporating frequent updates using big data tools.

• Exploring AI Implementation on Alternative Platforms: Questions arose about the implementation of LLaMa 2 on MLX for Apple Silicon, specifically on how to adapt the model’s query parameters to this platform. The technical intricacies of multi-head or grouped-query formats were under consideration.

OpenAI Discord Summary

• DALL-E Adopts Content Authenticity Initiative Standards: DALL-E generated images now include metadata conforming to C2PA specifications, aiding verification of OpenAI-generated content, as announced by @abdubs. The change aims to assist social platforms and content distributors in content validation. Full details are available in the official help article.

• AI Censorship and Authenticity Stir Debate: AI censorship's impact on user experience prompted a hot exchange; users @kotykd and @arturmentado respectively criticized and defended AI censorship, highlighting user freedom and misuse prevention. Ethical concerns about presenting AI-generated art as human-created were also voiced, emphasizing the need for honest disclosure per OpenAI's TOS.

• Metadata Manipulation Recognized as Trivial by Community: The significance of metadata in image provenance was heavily discussed, with users agreeing on the ease of removing such data, rendering it an unreliable measure for image source verification. This reflects the technical challenges of securing digital image authenticity.

• Open Source AI Models Positioned Against Commercial Giants: A debate flourished comparing open-source AI models to commercial options like GPT-4, touching on the impact on innovation and the potential growth of competitive open-source alternatives. The discussion reflects the engineering community's focus on the development landscape of AI technologies.

• Discussions Surround GPT-4 Usability and Development: Multiple users, such as @mikreyyy, @glory_72072, and others, sought help regarding GPT-4 usage issues like logouts, finding demos and storytelling capabilities. The conversation also touched on answer limits and complexities involved in custom GPT usage, indicating a concentration on the practical applications and limitations of GPT-4 in real-world scenarios.

• Community Seeks Improvement and Interaction in AI-Driven Projects: In the realm of prompt engineering and API discussions, community members @loier and @_fresnic looked for collaborative input on improving GPT modules and refining character interaction prompts. Meanwhile, @novumclassicum sought to provide feedback directly to OpenAI, indicating a desire for more streamlined communication between developers and the OpenAI team.

HuggingFace Discord Summary

• Hugging Chat Assistant Personalization: Hugging Face introduces Hugging Chat Assistant, allowing users to build personalized assistants with customizable name, avatar, and behavior. It supports LLMs such as Llama2 and Mixtral, streamlining the user experience by eliminating the need for separate custom prompt storage. Check out the new feature at Hugging Chat.

• Dataset Viewer for PRO and Enterprise: The Dataset Viewer on HuggingFace now supports private datasets, but the feature is exclusively for PRO and Enterprise Hub users. This update is aimed at enhancing data analysis and exploration tools (source).

• Synthetic Data Trends: HuggingFace Hub adds a synthetic tag to facilitate the sharing and discovery of synthetic datasets, signaling the growing importance of synthetic data in AI.

• AI-Powered Quadrupeds and AI in Fashion: Cat Game Research is developing a video game featuring the first quadruped character controller utilizing ML and AI, while Sketch to Fashion Collection turns sketches into fashion designs. Explore badcatgame.com and Hugging Face Spaces for innovations in AI-powered gaming and fashion.

• BLOOMChat-v2 Elevates Multilingual Chats: BLOOMChat-v2's 176B parameter multilingual language model with 32K sequence length is improving upon its predecessors. An API and further advancements are anticipated; details are communicated in a Twitter summary and detailed blog post.

• Reading Group Excitement and Resources: The HuggingFace Reading Group schedules a presentation for decoder-only foundation models for time-series forecasting, and a GitHub repository (link) is established to compile resources from past sessions, enhancing knowledge sharing.

• Diffusers and Transformers Learning: For those new to diffusion models, the community suggests courses by HuggingFace and FastAI for deep dives, while queries about details such as timestep setting and conditioning model validation suggest active experimentation and learning within the domain.

• NLP Channel Gets Textbook: A discussion on fine-tuning LLMs like Llama chat or Mistral with textbook content proposes to enhance domain specific understanding, thereby improving the educational chat capabilities of models.

LM Studio Discord Summary

• LM Studio Launches v0.2.14: A new LM Studio v0.2.14 version is released, tackling critical bugs like UI freezes and input hangs, which you can access via LM Studio website. Remember to update for a smoother experience.

• Ease-of-Use Shines with LM Studio: Users are drawn to LM Studio for its simple user interface, enabling anyone to use LLMs without coding skills. But, watch out for default resets in LLM folder locations after updates.

• Local Model Execution Challenges: While users experience issues like model generation freezes and poor GPU utilization, patches in LM Studio's latest updates are meant to address these. Also, for detailed model-fine tuning instructions, check out the YouTube tutorial.

• Hardware Hub: Debates continue over optimal hardware setups

Mistral Outshines in Programming:

• Carsonpoole discovered that Mistral outperforms phi2 significantly on the code section of OpenHermes 2.5 under identical sampling scenarios. The discussions included implications for GPT-4's programming capabilities and sparked curiosity surrounding the expected skillset of a 2-billion-parameter model, with cited expectations from Microsoft Research.

Sparsetral Unveiled and Math Benchmarking Excitement:

• The introduction of Sparsetral, a sparse MoE model, comes complete with resources such as the original paper and GitHub repos. Meanwhile, .benxh celebrated Deepseek, which incorporates a technique called DPO to achieve new proficiency levels in math-focused assessments.

Quant Tune and EQ-Bench:

• Tsunemoto has quantized Senku-70B, a finetuned version of the hypothetical Mistral-70B, yielding an EQ-Bench score of 84.89, and shared it on HuggingFace. This sparked a broader discourse on the significance of mathematics in appraising language models' abilities and hosting LLM-powered robotics hackathons.

Language Model Quirks and Mixtral Issues Noted:

• Users experienced that Mixtral, directed in Chinese, presents mixed-language responses, and similar issues with OpenHermes. Cloudflare’s AI platform adoption of these models was highlighted through tweets.

Support for Robot-Control Framework:

• Babycommando sought suggestions on finetuning multi-modal models and released MachinaScript for Robots with a GitHub repository. They asked for guidance on finetuning Obsidian and technical specifications for robot interactions using their LLM-driven framework.

Alignment Lab AI Discord Summary

• <strong>Seeking Synergy in Silicon</strong>: A user reached out for collaboration opportunities with the Alignment Lab project.
• <strong>Enthusiastic Newcomer Alert</strong>: Another user expressed excitement for joining the Alignment Lab AI Discord and offered expertise in cybersecurity.
• <strong>Ingenious Code Generation Tools</strong>: Introduction to AlphaCodium, a tool using adversarial techniques to generate high-quality code.
• <strong>Appreciation for AI Interview Insights</strong>: Commendation for insights from Jeremy's Latent Space interview.

Datasette - LLM (@SimonW) Discord Summary

• <strong>Free Hairstyle App Hunt Turns Fruitless</strong>: User hunting for a free hairstyle app faces challenges despite Google search efforts.

OpenAI announcements Discord Summary

• <strong>DALL-E Images get Metadata Upgrade</strong>: Announcement of image metadata addition in images generated by ChatGPT and OpenAI API following C2PA specifications.

• Troubleshooting Log-In Issues: User @mikreyyy inquired about logging out of all devices from their account but received no solutions in the discussion thread.
• Finding GPT Demonstrations: User @glory_72072 asked for guidance on finding and using the GPT demo, with no detailed responses provided.
• Experiencing Enhanced Storytelling: User @blckreaper expressed satisfaction with GPT's improved narrative output but reported issues with custom instructions.
• Queries About Answer Limits: User @ytzhak raised concerns about interaction limits with GPT, sparking a discussion on usage caps.
• Custom GPT Troubles and Tips: Users shared challenges and solutions with Custom GPT, from errors to instructional conflicts, and discussed cost-effectiveness of different GPT models.

The discussions in the HuggingFace NLP channel involved topics like fine-tuning LLaMA chat models on class syllabi, considering audio-to-text transcriptions for dataset creation, seeking clarification on dataset utility, encountering issues with fine-tuning models, and exploring the use of AI for domain-specific content comprehension and educational chat capabilities. Users also shared experiences with different models, troubleshooting model performance issues, and seeking advice on model preferences and hardware specifications.

• User Bug Reports:
  • User @goldensun3ds reported a bug where ejecting a model during message processing leads to the program hanging indefinitely. They suggested the need for a cancel generation feature to prevent program restart.
  • User @laurentcrivello flagged a persistent bug on LM Studio MacOS where the server remains reachable after app window closure.
• Feature Requests:
  • User @mike_50363 highlighted the lag in the non-AVX2 beta version, impacting usage on certain systems.
  • User @laurentcrivello proposed optimizing the UI for server indication with a preference for fewer active app indications.

Mistral Finetuning

• Padding Conundrums in Fine Tuning: User expressed confusion about generating text with padded tokens.
• Seeking Clarity on Fine Tuning Practices: User shared a tutorial used for reference on fine-tuning language models.
• Fine-Tuning Platform Recommendations: User sought advice on platforms for fine-tuning after encountering an error.
• Mistral Training Questions: User asked for guidance on fine-tuning Mistral v0.1 7B beyond 8k tokens.
• Clarification on Model Pretraining: User raised a concern about using 's2_attention' with Axolotl and sample packing, and how it relates to Mistral v0.1 7B's pretraining methodology.

Links mentioned:

• Fine-tuning Language Models for Structured Responses with QLoRa
• Discord - A New Way to Chat with Friends & Communities

Discussing Infinite Depth Limit

• @niket mentioned the relevance of work on the infinite depth limit by researchers like Greg Yang and Soufiane Hayou.

Expanding on Tensor Programs

• @niket referenced tensor programs, possibly the sixth iteration, in the context of deep learning and scaling laws.

Exploring Loss Landscapes in Scaling Laws

• @niket expressed an interest in understanding how the loss landscape behaves at these infinite limits, noting a lack of resources and requesting references for existing research in this area.

The section covers various discussions related to artificial intelligence in different Discord channels. Users discuss topics such as using LLMs and RAG for enterprises, PDF parsing with LlamaIndex, RAG documentation by MistralAI, and more. In another channel, users share advice on optimizing RAG system performance, asynchronous blocking in agent creation, and integrating Langchain with AWS SageMaker. The section also includes conversations about AI personal assistants, programming with AI, RPA, exploring vector databases, and GPU monitoring tools. Users also highlight tools like CLTune for OpenCL and CUDA kernel tuning and GMON for simplified GPU monitoring.

This section covers discussions on various topics in different Discord channels. Users seek advice on GPU alternatives for MacBook Pro, recommend cloud resources like Google Colab, discuss Jax rising in popularity, share dataset recommendations for training, address issues with Jina embeddings, report OOM issues with Axolotl, and suggest Deepspeed configuration caution. Other discussions involve handling PDF documents with OCR, creating a two-tier pricing model for German inference services, introducing Super JSON Mode for low latency output generation, seeking better hosting for MythoMax, and exploring collaborations in AI projects.