šŸ“± ā€œSlimmedā€ Llama Models for Edge Devices, From Meta

Meta introduces new, compact Llama models tailored for devices with limited power, enhancing AI capabilities while reducing memory usage.

Author

Jack Lajoie
October 25, 2024 ā€¢ read time ~ 8 minutes

The Daily Current āš”ļø

Welcome to the creatives, builders, pioneers, and thought leaders ever driving further into the liminal space.

Meta's new slimmed down Llama models further exemplify the trend towards creating optimized AI for ā€œedgeā€ (personal) devices, reducing reliance on cloud computing. We also have more legal drama between Perplexity and publishers, while Meta's collaboration with Reuters seems to represent a proactive step in solving that conflict. OpenAI's articulation of its ethical framework for national security partnerships demonstrates a commitment to responsible AI use.

 šŸ”Œ Plug Into These Headlines:

Meta has released quantized versions of its Llama 3.2 1B and 3B language models, optimized for low-powered devices. These slimmed-down models maintain strong performance while reducing memory footprint and increasing inference speed, making them suitable for on-device and edge AI deployments. The quantization techniques used, QLoRA and SpinQuant, have resulted in significant improvements in size reduction and processing speed.

  • Average 56% reduction in size compared to original formats

  • Inference processing speed increased by two to four times

  • 41% reduced memory usage on average (tested on Android OnePlus 12 smartphones)

  • Optimized for Arm-based SoCs through collaboration with Qualcomm and MediaTek

  • Available for download from Llama.com and Hugging Face

šŸ’” The release of these quantized models highlights the growing importance of edge computing in AI, as companies seek to balance performance with privacy and reduced reliance on cloud infrastructure.

Perplexity AI responds to News Corpā€™s lawsuit by highlighting the value of AI in enhancing information retrieval and comprehension. The company refutes claims of improper use of copyrighted material, pointing to its transparent citation practices and revenue-sharing initiatives with other publishers. While preparing to defend itself legally, Perplexity maintains that AI-driven search tools are an inevitable and beneficial evolution in how people interact with information online.

They prefer to live in a world where publicly reported facts are owned by corporations, and no one can do anything with those publicly reported facts without paying a toll

Perplexity Team

  • Itā€™s more of a public ā€œclapbackā€ than a response with true legal substance, but itā€™s still interesting to see Perplexity allude to how they plan to fight this fight

  • They emphasize their practice of listing sources and providing in-line citations for answers.

  • Perplexity has already launched a revenue-sharing program with publishers like TIME, Fortune, and Der Spiegel.

ā±ļø Itā€™s a bit ironic to see Perplexity become the villain figure cast against the news media world, as they were originally cast as the AI tool making sincere efforts to avoid running afoul of publishers (through their citations). But at the end of the day, most users arenā€™t going to click the links they provide, preferring to read their brilliant summaries instead. Might be too little too late for publishersā€¦

This partnership will leverage Reutersā€™ vast news archive and real-time updates alongside Metaā€™s AI chatbot. The project aims to allow users of Meta AI to receive real time news updates and conduct discussions on current events.

  • The agreement includes a multi-year licensing deal for Reutersā€™ news content, including its archive and real-time updates.

  • This collaboration aims to combat misinformation and provide reliable news sources in the digital age.

  • The chatbot is expected to offer personalized news experiences for users, leveraging Metaā€™s AI capabilities.

  • Reuters will maintain editorial independence in its news coverage throughout this partnership.

šŸ¤” Looks like Metaā€™s been doing some soul searching on its role in public discourse. After scaling back news and political content this year, theyā€™ve decided to ā€œaddress misinformationā€ by creating a hyper-personalized version of news. Reuters is at least well regarded as a reliable news outlet.

In response to the White Houseā€™s National Security Memorandum on AI, OpenAI outlines its position on AI in national security contexts. The company advocates for a balance between advancing AI capabilities and maintaining ethical standards. OpenAIā€™s framework for assessing national security partnerships is guided by four key values: Democratic Values, Safety, Responsibility, and Accountability.

  • OpenAI states that it supports U.S. and allied efforts to advance AI while upholding democratic values in AI leadership.

  • The companyā€™s usage policies prohibit using their technology to harm people, destroy property, or develop weapons.

  • OpenAI has developed a framework for assessing national security partnerships, guided by four key values: Democratic Values, Safety, Responsibility, and Accountability.

  • All potential national security partnerships undergo a rigorous internal review process led by OpenAIā€™s Product Policy and National Security teams.

šŸŖ– We might be reading into this a little too deeply, but OpenAI looks to be intending to identify itself as the America-first AI company while also distancing itself from what could easily become military applications.

Vice President Harris is generally in favor of constructing a regulatory framework for responsible AI development, while Trumpā€™s team advocates for minimal intervention to foster innovation.

  • Both sides acknowledge the need to maintain U.S. leadership in AI development, but differ on methods.

  • Trumpā€™s campaign criticizes the Biden approach as potentially stifling innovation and economic growth.

  • AI policy is becoming an increasingly important issue in the 2024 presidential election.

šŸ” The AI policy debate (or lack thereof, more accurately šŸ˜†) in the 2024 election underscores the frequently surfacing fundamental tension between regulation and innovation, reflecting broader societal concerns about the rapid advancement of technology.