OpenAI “Spring Updates”

OpenAI announced significant updates to ChatCPT and the GPT-4 family of models, namely introducing GPT-4o (“o is for “omni”, apparently). The event was a different format than we have seen in the past from OpenAI (especially without Sam Altman presenting), and I encourage you to watch the replay.

GPT-4o is not a completely new model (i.e. GPT-5), but instead a significant update to the GPT-4 model. While there was certainly speculation (and perhaps an expectation) that a brand new model was going to be released (GPT-4 launched in 2022), this update is quite significant. We tend to be caught up the technical nature of the LLM itself (especially model size), but this was truly a product update versus just a model update. Speed was certainly the key theme on display here.

For example, Voice Mode in GPT-3.5 had latency of 2.8 seconds and GPT-4 had latency of 5.4 seconds. In GPT-4o, the model can complete a conversational interaction in as little as 232 milliseconds. OpenAI covers more details around this speed increase in their blog post here. Outside of the model performance, the previous ChatGPT Voice Mode user experience was quite clunky as it required you to fumble through the app. That iteration was good enough to show the vision, but this update makes this much closer to a compelling product. There are certainly still issues with the model responses, the model “personality” (including a mini-scandal with Scarlett Johansson over similarities to her voice), and a glitchy demo (even with wired connectivity). I’m willing to overlook these shortcomings for now as a fairly significant step change forward in getting to a true assistant that you can actually talk naturally with. We aren’t quite at Her-level interactivity or anything resembling AGI (yet).

The other notable update is that this is available to all free users. The vast majority of ChatGPT users are still at the free tier, and have been using GPT-3.5 Turbo up until now. This is an enormous leap forward for those users, and speaks to the confidence of OpenAI’s ability to scale GPT-4o and their infrastructure. While inference costs are slowly going down, I’m particularly interested in seeing how OpenAI will be able to continue to financially support the free tier. There also does not seem to be a great value proposition for the paid tier now, which is probably a signal a significant update is coming soon for paid users (perhaps GPT-5). OpenAI is making good progress in the enterprise space, but seemingly still needs to figure out the consumer business model. I can’t imagine this can be the loss leader for the company forever.

Google I/O

Google’s annual developer conference comes just a month after the company’s enterprise keynote, but unfortunately fell a bit flat coming off of the heels of the OpenAI update.

Starting with the positives, all of the updates to infrastructure and existing products were quite compelling and are shipping in the coming weeks or months. Google’s approach to integrating AI with data inside existing products or environments was particularly impressive. A few announcements that I found interesting:

• Gemini: Significant updates to Gemini 1.5 Pro and new version, Flash, are available immediately. The 1M token context window is already incredibly powerful, and the company announced a 2M token context window model will be available in preview for both model variants. Gemini 1.5 Flash promises to be both faster and 1/10 the cost, which should be a boon for quite a few use enterprise use cases.

• Search: Updates that include Gemini-created “AI overviews” and grouping of results will be rolling out in the next few weeks in the US. I’ve been impressed using the Search Generative Experience (SGE) preview for a while now. Extending SGE with the announced multi-step reasoning, Search Engine Results Page (SERP), etc. is a nice improvement that doesn’t feel like nifty features haphazardly bolted onto search. I applaud the confidence to roll this out broadly. It also exemplifies the infrastructure advantage Google has to do this at scale. Search continues to be Google’s core business, and they don’t seem to be afraid to continue to evolve the product in a meaningful way.

• Gmail: The demo with Gemini was something I could see using immediately myself. Impressively, everything in the demo happens within Gmail and the mobile app to summarize and compile documents with all of the data that resides within Gmail. Compiling a variety of documents, emails, and other attachments are real life use cases I have on a daily basis in both my personal and professional life. Doing this kind of thing on a tiny smartphone screen would be challenging at best. I believe some of this is still done in the cloud, which also speaks to the importance of vertical integration there.

• Android: This demo was impressive with Gemini working at the system level in the device to do a variety of tasks like searching a PDF without having to jump between apps. Technically impressive and incredibly powerful from a usability standpoint. I would expect Apple to announce some similar capabilities in iOS at WWDC, but won’t imagine it will be as impressive (yet).

• Generative Media: This segment showed some great promise around music production that actually facilitates the creative process instead of just regurgitating training data like many other tools out there.

On to the negatives, much of the rest of the event was a series of hand-wavy concept videos. Project Astra is a very similar concept to what OpenAI actually demoed, but the Google project has a roll out planned at some point later this year. Similarly, the AI Agents part of the presentation was seemingly barely a vision (unlike what Microsoft demoed this week). Google is famous for showing off concept videos in their events, but when you see a live demo like OpenAI (flaws and all) the same week the message doesn’t seem as compelling. Personally, I would have not included any of the products that aren’t shipping in 2024 in the presentation.

Google’s strength, as evidenced by the product updates shipping soon, is really in the engineering prowess and the infrastructure capability to create performant, integrated systems at scale. The enterprise keynote last month truly showed that off, but this presentation veered too far off into concept especially when compared to OpenAI and Microsoft’s presentations. To that point, any actual new product announcements were curiously absent from I/O. LINK

Microsoft Windows AI & Build

In my opinion, Microsoft had the most impressive announcements this month. The company covered a myriad of areas from hardware to services, and most of which are all either available now or shipping soon.

• Copilot+ PCs: This is an enormous step forward to support AI workloads in the Windows ecosystem especially with on-device inference and native integrations. These new machines are Arm-based utilizing the new Qualcomm Snapdragon X chips instead of x86 Intel chips (though Intel and AMD chips will be in other devices moving forward apparently). Microsoft announced their own devices (new Surface and a tablet) and those from many OEMs you would expect (including Dell, ASUS, Samsung, Acer, Lenovo, and HP). The shift from x86 to Arm is a huge signal, and a perhaps a sign of the slow death of x86 computing. Microsoft has been attempting to support Arm for around a decade at this point, but now seems to be the time to shift akin to Apple’s transition a few years ago (especially if Qualcomm can scale up more and developers jump to support Arm-native apps). Speaking of Apple, the comparison to Apple’s MacBook Air were very direct including a head-to-head live demo as part of the keynote with the Surface machine clearly the winner (Microsoft claims it is 58% faster than the Air M3). On that note, interestingly the current generation Air does not quality as a Copilot+ device because the neural processing unit (NPU) is not performant enough. I haven’t seen any of the devices hands-on yet, but they look great and make me excited about Windows hardware again. The tablet in particular with the removable keyboard looks particularly well designed, and in many ways is what I wish Apple had done with the Magic Keyboard for the iPad Pro. LINK

• Recall & Windows Semantic Index: New feature within Windows that allows AI-powered photographic “memory” that lets users access basically everything they may have seen on the device. Importantly, this is all processed on-device (hopefully reducing the privacy risk), and an example why this new generation of hardware with deeper software integration is needed. I’m excited to test this feature to see if it’s actually useful. LINK

• Copilot: Major updates to the generative AI services within the 365 suite and Teams that include features like note-taking, chat moderation, agents, and contextual Q&A. The agent experience seems to be quite useful, and the examples provided (from data entry to more complicated tasks like project management) are all real improvements to the working life of many professionals. Microsoft said this won’t completely displace jobs, but instead simply take away the these menial tasks. While I tend to agree most roles won’t be 100% impacted here, there is so much swivel-chair work still out there in every organization that could simply go away if this works better than most other automation today. I’ll be excited to evaluate when it hits Copilot Studio later this year. LINK

• Azure AI & Models: Microsoft announced an update to the Phi-3 AI model family called Phi-3-vision. This new model is multi-modal with focus on reading text and interrogating images, but importantly is small enough that it should work on mobile devices. Lots of use cases need this type of functionality, and seems to further validate the need for models of all sizes. The company also announced updates to the Azure OpenAI service including availability of GPT-4o, fine tuning for GPT-4, Assistants API, etc. All of these are nice updates to what is already a compelling offering through Azure. LINK

• Advanced Paste: New feature within the PowerToys suite inside Windows 11 that allows you to convert items within your clipboard and invoke OpenAI through a prompt window. To use the AI function, you need to have credits in your OpenAI account. The feature itself is somewhat interesting, but for me the bigger takeaway is the signal of how pricing models may work for AI-enabled features. LINK

Microsoft has been positioning themselves as the AI platform player, and with the huge steps forward in their hardware portfolio and tighter integration with their software and services that vision seems much closer to a reality. Also, many of these announcements are Microsoft first-party offerings, but the partnership components (OEMs for the Copilot+ hardware, OpenAI, Khan Academy, etc.) and positioning should enable some significant scale and reach advantage Google and Apple haven’t quite achieved yet with their respective approaches.

Apple rumors

The UX barrier for third party AI assistants, even with this GPT-4o update, is that you still have that step to unlock your phone and launch an app to interact with it. Apps also doesn’t necessarily have any deeper integration with the other data or services on the device. Apple has been not-so-subtly focused on readying their hardware for all things AI especially for on-device inference, but need some partner to provide the LLM and “killer apps” still given the state of its in-house models.

There have been many rumors that Apple was negotiating a deal with Google to provide Gemini within the iOS ecosystem (especially in light of the search partnership), but now it seems OpenAI is the clear frontrunner. This could be a move to keep OpenAI closer and avoid medium-to-long term competition around hardware that has been rumored. This also could be a clever move, much like the Google search partnership, to shield Apple from any potential issues (like hallucinations, privacy, etc.) with the model itself. It could be all of the above. I’m not sure OpenAI, especially given the myriad of controversies, is the best fit for such a risk-averse company like Apple. If not Google, Microsoft feels like the more stable, mature bet personally. It is unclear if Microsoft even wants to look at that kind of partnership even though they have a significant blind spot in the mobile space.

Mark Gurman wrote a piece this week about what we may expect from the WWDC announcements, including some nifty capabilities across the core apps within iOS and macOS including software to route specific tasks to process either on-device or in the cloud. This all sounds like catch-up so far.

Key questions come to mind already: What is the business model and structure for this kind of partnership? Is this some sort of revenue share? Does Apple open up the ability to select the “default” LLM/assistant outside of Siri?

I am expecting to see some big announcements at WWDC this summer to help shed light here, but I would not bet on anything that transcends. I can’t remember a WWDC with more expectation especially given the competitive narrative these days. LINK

Key takeaways on the “AI Platform Shift” [TL;DR]

In historical platform shifts, there has been fundamental technological innovation followed by a distribution shift that together enables reorientation of markets and value chains. In the mobile example, websites initially turned into apps, but real disruption didn’t happen until companies did things you could only do with everything else that came with the mobile devices (e.g., sensors, GPS, cameras, connectivity, etc.) to leverage in specific markets (e.g., Uber and the taxi industry). That distribution and value aggregation shift usually takes time to manifest.

There are phases to the process, and we are still in the early innings of the Generative AI story despite over a decade of machine learning. Generative AI is certainly here to stay, and is becoming a part of every software application and modern tech product. We seem to be moving further into the phase where we’re better understanding the technology and what it may be useful for. Incumbents are incorporating features into current products and continuing to exert their existing strengths in infrastructure, scale, and distribution. Those incumbents are attempting to build the foundational integrated platform components that likely allow them to capture a significant part of the value chain for now. Upstarts haven’t quite cracked how to unbundle pieces of the existing value chain yet, but that will likely be part of the next phase here.

Here are some of my key takeaways from the announcements this week:

• Model performance: Multi-modal and general purpose model performance is important to many of the current use cases. Smaller, more efficient models seem to be finding an important place within these systems as well.

• Speed: Speed is becoming the critical differentiator, especially for the general purpose, conversational AI “killer apps”. This speed does not simply come from the underlying model itself, though certainly an important part of the system.

• Infrastructure & scale: Incumbents are continuing to leverage existing infrastructure and scale strength to great effect. Existing reach and distribution is helping solidify positions by introducing AI in a meaningful way into existing scaled products.

• Hardware integration: A new generation of end user hardware is enabling more integrated, on-device AI experiences. This tighter coupling at the device level in concert with cloud services should enable more of a true “platform” and ownership of a significant portion of the value chain for now.

• Partnerships: Upstarts are pushed to partner even if the product or technology is truly differentiated as the “platform” becomes more vertically integrated by the incumbents. Applications like chatbots don’t have to live within the “platform” per-se (and can likely be a standalone app), but being embedded into the core platform could provide competitive advantages.

In light of these themes, each company is in their own place at the moment:

• OpenAI: One of a few traditional “startup” examples (hard to call a company with a $86B valuation a startup) in the space today with a compelling set of unique offerings. It seems partnerships may be the short-medium term pathway to continued growth.

• Google: Seemingly forging their own path across the entire AI ecosystem with a limited partner ecosystem. The quintessential vertical integrator leveraging their infrastructure and scale strengths.

• Microsoft: The most compelling provider in the market today that leverages infrastructure, scale, first-party offerings, and meaningful partnerships. The developer experience seems to be the most “complete” to simply build your app within their platform. However, Microsoft is still missing a mobile angle.

• Apple: The foundational elements of a platform are there, but it seems evident partnerships will be needed. The gauntlet has been set to catch up with significant pressure on this year’s WWDC to shed light on the vision to do so.

The technology shift around AI is happening at a breakneck pace, but there is still a long road to truly tell the magnitude of the platform shift. Right now, Microsoft seems to be in the best position to navigate this shift and capture the majority of the value of the incumbents today. We still have some key questions still to be answered:

• How much better do models get?

• Does a competitive moat reside with the models or model providers themselves?

• Can you have a new platform if you don’t have hardware?

• Can new business models truly emerge?

• Are there network effects?

• Can you aggregate distribution in a meaningful way?

Earnings

Apple (NASDAQ: APPL)

• 2Q revenue reported at $90.75B (down 4% YoY). Net income was also down 2% YoY to $23.64B.

• iPhone revenue was down 10% YoY to $45.96B; iPad revenue was down 17% YoY to $23.9B; Services (including Apple Music, Apple TV+) revenue was up 14% YoY to $23.9B

• The company announced a $110B share buyback which is the largest share buyback by any company in history. If Apple chooses to buy back all of the shares it has said it would, it would own roughly 4% of its own business.

• China sales were a significant story up until earnings, but it seems revenue in the region was reported at $16.4B. This was down only 8.1% YoY compared to some analyst estimates showing reduction as much as 19%. The decline, even though it is not as significant as expected, is still material. I would be particular interested to see the revenue breakdown between iPhone vs other device vs services. I have to expect services and other device revenue lagging more than the iPhone. If this is the case (especially on the services front), I would expect the longer term outlook in China to be more of a concern.

• Despite less than exciting quarterly results, the share buyback and dividend increase (up 4% to $0.25/share) news helped lift the stock 5% after hours. Services revenue growth still seems to be a bright spot here, and I expect some good momentum through the rest of the year with multiple device refreshes and impending AI news at WWDC next month.

• While not directly earnings related, a story came out in Bloomberg this week on potential successors to Tim Cook. John Ternus, the current head of hardware engineering, seems to be the rumored frontrunner. LINK

• LINK

Arm (NASDAQ: ARM)

• The chip designer’s quarterly results crushed expectations with 4Q revenue of $928M (up 47% YoY).

• The licensing business grew 60% YoY to $414M driven by AI chip demand. Royalties also grew significantly, up 37% YoY to $514M.

• Unfortunately, guidance fell below analyst expectations sending the stock trading down on the news. While the guidance and full-year earnings seemed to be mostly in-line with expectations, the forward price/earnings multiple is around 70, which is certainly very high (Nvidia is around 40 for reference).

• While I do think the China risk the company called out in their F-1 is still real, Arm is right at the center of the data center and handset growth driven by AI and should benefit handily.

• LINK

Qualcomm (NASDAQ: QCOM)

• The company reported 1Q adjusted revenue at $9.4B (up 1.2% YoY) and net income at $2.3B (up 36.5% YoY). Handset chip sales were reported at $6.2B (up 1% YoY).

• The Android market in China appears to be still very strong with revenue from Chinese phone manufacturers (like Oppo, OnePlus, Vivo) growing 40% YoY.

• There was a lot of talk about how the company’s chips are enabling the next generation of on-device AI capabilities, though aside from Samsung’s Galaxy S24 Ultra there aren’t a ton of examples yet.

• The US also just announced after Qualcomm’s earnings the licenses for Intel and Qualcomm to sell chips to Huawei have been revoked which I expect will likely impact the next few quarter’s results. LINK

• LINK

Palantir (NYSE: PLTR)

• The company posted revenue of $634M (up 21% YoY) and net income of $105.5M. Palantir has posted a net profit for six straight quarters now.

• Despite the solid results, weaker-than-expected full-year guidance sent the stock down 7%. Another example of a company reporting strong revenue and EPS, yet leading to a drop in the stock price this earnings season. As from the last few quarters, growth seems to be shifting from government to private sector.

• In his letter to shareholders, CEO Alex Karl said of customers, “They need results now. And we believe that we have the only platform that works.” LINK

• LINK

Apple “Let Loose” Event

iPad Air: The biggest news here is the introduction of a 13” iPad Air (finally). There is more trickle down from the iPad Pro to the Air lineup, but not a lot truly new here.

iPad Pro: As expected, the iPad Pro was the focus. It is the thinnest thing Apple has ever produced, and apparently would not have been possible without the efficiency of the M4. Up until this 5.1mm M4 iPad Pro, the iPod Nano (7-gen) was the previously thinnest Apple device at 5.4mm. It is also much lighter than the previous iPad Pro (by roughly 100g). Battery life is basically the same as the last generation and is still nowhere near the batter life of the MacBooks (my biggest complaint with my iPad Pro). The other big announcement was the introduction of the new tandem OLED Ultra Retina XDR display that Apple claims is the “worlds best display”. The new Magic Keyboard is now made of aluminum (finally) and looks very premium.

Pencil Pro: There are now an assortment of Pencils in the lineup with a new Pencil Pro announced this week. Apple now has the Pencil Pro, Pencil USB-C, Pencil 2, and Pencil 1. Curiously, the Pencil Pro is compatible with both the M4 iPad Pro and M2 iPad Air. Traditionally, the Pro naming convention was exclusive to the Pro products in the lineup. The new Pro supports some more gestures and provided haptic feedback. Personally, I use the Pencil very infrequently, but for those that draw a lot I’m sure this is a nice update.

M4: As Mark Gurman correctly predicted, the M4 was unveiled this week in the new iPad Pro. As I discussed briefly last week, there has been a major push to get to a more regular cadence on the M-series chips. It has been reported for a while that Apple and TSMC has been trying to move as quickly as possible to the new 2nd generation 3nm process. Apparently the first generation of the 3nm process used for the M3 simply wouldn’t scale, and those chips were prioritized for the MacBooks. The M4 has 4x faster GPU performance than the M2 and 50% CPU than the M2. Curious the comparison was not to the M3, so we’ll need to wait for some real-world benchmarking here.

AI: Between the earnings call and the “Let Loose” event, Apple has been talking A LOT about how AI is coming to everything. Unfortunately not much beyond the hardware that is supporting these features has been announced as of yet. We will likely need to wait until WWDC this summer for anything truly significant.

“Should I buy a new iPad?”: I have received a number of questions from friends and family so here is an abbreviated buyers guide:

• iPad Air: If you have been waiting to upgrade your iPad and do not need any of the Pro features, this is a great opportunity to refresh. The 11” starts at $599 for the 128GB model so it is good value for a very solid device. I’d recommend upgrading the storage to at least 512GB. If you are buying it for Mom or Grandma to mostly message and FaceTime the family, the 128GB model should be just fine. Mom may benefit more from the Air than Grandma. Grandma would likely be very happy to have the regular iPad. Both would also appreciate if you called them more often.

• iPad Pro: If you are trying to make this your primary computing device to do real work, I would not recommend spending the money here and wait for the MacBook updates later this year. I have a 2022 12.9” iPad Pro that has turned into my preferred device for travel over my MacBook Pro IF I’m not planning on doing real work. It’s an excellent device for light work with no real multi-tasking, but unfortunately is very frustrating if you are trying to run multiple apps. Split View, Slide Over, and Stage Manager have unfortunately not offered the level of usability I would expect. Also, annoyingly, many Apple apps like Notes are just better on Macs still (there is still no calculator on iPad for some reason). If price is no issue and you want something to read email, watch streaming video, do some quick photo editing, etc. on the go then the iPad Pro is an excellent choice. I reserve the right to change this recommendation when more of the AI features launch after WWDC.

VERGE; ENGADGET

AI

Microsoft MAI-1: The Information had a piece this week on development of an in-house AI model with ~500B parameters that should make it able to compete with other top models (for reference, OpenAI GPT-4 has 1.76T parameters). Microsoft has been very quick to invest when necessary to be in a leadership position here, and this may be one of the first significant outputs from the company’s deal with Inflection. It was only a matter of time before Microsoft started to vertically integrate the LLM. While I still feel that closed, general purpose LLMs don’t offer a lasting competitive advantage, it is almost table stakes today for large companies to offer their own. Microsoft can certainly monetize the model through the existing service portfolio at a higher margin than some of the partner models. LINK

Anthropic Team & iOS app launch: Anthropic has now launched a new enterprise plan (Team) that provides access to the company’s models (Claude 3 Opus, Sonnet, Haiku) for $30/user/month. Microsoft offers a Copilot subscription for their 365 suite of products for the same price (on top of base 365 plan pricing), and it seems enterprises have been trying to justify that pricing (LINK). Interesting to see how many early GenAI services are web first and then native apps start to appear. The go-to-market approach to package up an enterprise solution with centralized management, billing, security posture, copyright indemnity, etc. absolutely makes sense, and follows Anthropic’s approach to serve enterprises over consumers. I still am not entirely sold on closed, (mostly) general purpose LLMs being a long term differentiator. Having said that, the 200k context window size across their entire model lineup with reasonable performance is exciting right now especially for software engineering use. LINK

AlphaFold 3: Google DeepMind and DeepMind spinoff Isomorphic Labs announced details around a new version of their model to generate 3D structures of molecules, allowing prediction of interactions and structures like DNA, RNA, proteins, and antibodies. AlphaFold originally launched in 2018 and last updated in 2020. This third iteration now supports more than just proteins which is perhaps one of the largest recent AI breakthroughs in the biotech space. Demis Hassabis (CEO of Google DeepMind) is looking at AlphaFold and related tech to drive “enormous commercial value” and a $100B+ business. Isomorphic Labs complements this model with a collection of other closed models and tools around drug design. I do feel specialized models and a business focused around a particular industry can be truly disruptive vs general purpose LLM-based apps. It helps to have the resources and runway of Google to support the R&D without the short-term ROI requirements. AlphaFold 3 is available publicly on AlphaFold Server for non-commercial use. LINK; BLOOMBERG

Potential AI model China export regulation: There are rumblings that the US Commerce Department is looking to define regulation around export of closed source AI models to China. While there is significant restriction around the export of chips and other related hardware, there is currently (almost) nothing stopping US AI companies selling software or services to anyone. There is already quite a gap in the hardware available to Chinese companies to train leading-edge models from the last few years of regulation, so it is no surprise access to top models trained in the US would be next. There is still a lack of any cohesive AI legislation in the US, but I expect the China dynamic with focus on national security to contribute significantly more than the EU’s legislative approach that focused more on consumer protection. LINK

Amazon Q: The company launched a new LLM-based assistant focused on coding this week. Everyone is getting into the coding copilot business which speaks to both the applicability of this kind of technology to software engineering and the customer desire to optimize productivity and cost across engineering teams. There is significant investment in this space on the venture side (see last week’s newsletter), but more to be told how differentiated any of these offers may ultimately be. LINK

Washed Out’s ‘The Hardest Part’: The first commissioned music video made entirely with AI (OpenAI’s Sora) by a major music label. Very impressive work that does open more conversation on the long-term viability of AI-generated art. I would love to see the prompts used and how much this video actually cost to produce given the length and quality. LINK

M&A & Venture

CoreWeave: The company, which is a specialized cloud provider delivering Nvidia GPU resources, raised a $1.1B Series C round lead by Coatue valuing the company at $19B. The company last raised their Series B last May that valued the company at $2B. CoreWeave originally focused on crypto workloads, but has pivoted well to focus on AI. Competition is growing, however, which is likely why they looked to raise another large round here a year after their previous round. AI funding has slowly been ticking up here with a 4% increase from 4Q23 according to Crunchbase. I expect continued significant growth on the infrastructure side in particular as the software and end user application ecosystem shakes itself out. WSJ; CRUNCHBASE

SoftBank in talks to acquire Graphcore: SoftBank is rumored to be in advanced talks to acquire Graphcore. Graphcore is a UK-based chip startup that was once valued at $2.8B, but has been struggling recently and reported just $2.7M in 2022 revenue and reported recent layoffs. The company was originally seen as significant competition to Nvidia, but unfortunately has been fighting market conditions that have stifled growth of its Intelligence Process Units (IPUs). While the core technology is certainly novel, the company has really had a tough time building a partner ecosystem. Even if SoftBank does purchase Graphcore, I feel it will be quite difficult to find an exit like Arm here. LINK

Silicon Valley Bank (SVB) Capital sale: SVB has agreed to sell its venture business SVB Capital to a new company backed by Sequoia Heritage and Brookfield for roughly $340M (all cash) pending court approval. SVP Capital is the last remaining major operating unit of the bankrupt parent company and has roughly $9.8B in assets under management. This should give the unit a fresh start, mercifully. LINK

Wayve: The self-driving startup based in the UK raised a $1.05B Series C led by Softbank with Nvidia and Microsoft also participating. Wayve utilizes an approach to learn from human behavior they call “Embodied AI” with multi-modal models (known as LINGO and GAIA), and is planning to commercialize to multiple auto and robotics manufacturers manufacturers. No significant customers have been announced yet, but the addressable market is enormous when you look at the landscape of auto manufacturers without anything resembling “mature” self driving tech outside of Tesla. Interestingly, this is the largest AI raise in the U.K. to date. The Nvidia investment and technology partnership is notable here, especially as the next generation of Wayve tech utilizes Nvidia’s Blackwell architecture, DRIVE Orin, and DRIVE Thor. LINK

Silicon

US chip manufacturing growth: The Semiconductor Industry Association (SIA) and Boston Consulting Group published analysis supporting a tripling of US domestic chip manufacturing capacity by 2032. This would bring market share up from 10% today to 14%. This would be the largest increase over the same time period in the world, but also speaks to the long road to acquire significant market share on-shore. Interestingly the report stated that without the CHIPS Act, the US market share would have fallen to 8% by 2032. Overall an early positive sign of the benefit of centralized funding and incentives for private industry in the sector, but there are significant challenges to actually get to that kind of scale including labor and operating costs in many states in the US. The CHIPS Act’s initial $39B infusion will most certainly need follow-on support at the federal, state, and local levels to keep up this initial momentum. LINK

Legal

US v. Google: Judge Amit P. Mehta is presiding over arguably the most important tech antitrust case since the 1990s, and grilled both Google and the U.S. prosecutors in the closing arguments this week. The NYT had some good coverage of both the closing arguments and a profile on the judge himself. LINK, LINK

Also, newly unsealed documents from the US Justice Department’s antitrust lawsuit against Google around the company’s search and advertising businesses have given some more context to a few items I found interesting:

• Google paid Apple $20B (2022) to be the default search engine in Safari, and this relationship is now roughly 20% of Apple’s operating income by itself. Rumors pegged the payment at something quite large, but having the actual number released coinciding with Apple earnings this week does put the scale into context. Apple had a total of $388B of revenue and $95B of net income in 2022. Without the Apple ecosystem and Safari, there is no Google deal. Google clearly can monetize this extremely well, and feels like a revenue share. LINK

• Emails from Kevin Scott (CTO at Microsoft) in 2019 to Satya Nadella and Bill Gates talking about how concerned he was over the gap in AI capabilities to Google. “And as I dug in to try to understand where all of the capability gaps were between Google and us for a model training, I got very worried.” This ultimately seemed to help spur the investment in OpenAI and further investment in the larger GenAI ecosystem (see below for some more MSFT news this week on their own LLM work). While Microsoft has certainly been visionary in other areas in the past, sometimes competitive fears drive more action than altruism or innovation. LINK

Licensing copyrighted data: Companies are starting to sign deals with news sites and other copyright holders to train their models and/or provide AI-generated content, but there is even more litigation coming for those that do not. As discussed lat week, I still do not expect we see any real outcomes from this litigation for quite some time or legislation in the US or abroad to address copyright appropriately.

• Google signed a deal with NewsCorp LINK

• OpenAI signed a deal with Financial Times (FT) LINK

• Alden Global Capital (owner of Chicago Tribune, Orange County Register, and 6 other publishers) is suing Microsoft and OpenAI LINK

• UK legislators call the government approach to copyright related to AI and LLM development as “inadequate and deteriorating” LINK

State-specific AI legislation: Interesting research on a variety of AI bills being proposed across 10 states to address the technology in a variety of contexts. While seemingly a meaningful step forward, many of the bills contain some fairly significant loopholes that would limit any accountability at all. Unfortunately many of these bills stand a real chance at being passed this year which would likely provide little or no real governance. These states are trying to do the right thing, but sadly are missing the mark and may lead to a patchwork of ineffective regulation similar to the state-specific privacy laws we have today. LINK

Other news

Microsoft security overhaul: Off of the heels of a variety of security issues (to put it mildly), the company is overhauling its security culture and tying executive compensation to performance. For better or worse, Microsoft can be a single point of failure for many organizations including the federal government. This evolution will take some time as many business units within the company are rather fragmented. Trust is critical especially for the largest companies in the world using Microsoft’s software and services, but I do not expect significant material impact to Microsoft’s business in the short term even if there are additional breaches due to how complicated it would be to actually transition away from their services. LINK

Tech pandemic hangover continues: The Financial Times published a story this week detailing how 50 companies that thrived during the pandemic have since lost $1.5T in market value since 2020. Twilio, Zoom, Longi, Sea Group, Peloton, and RingCentral have had some of the biggest swings. The lesson here seems to be to never underestimate how market conditions like interest rates, consumer spending, return to office, etc. can change, and companies should be nimble enough to regress back to the mean quickly. It is a good reminder profitability trumps growth and to not get caught up in the moment and assume external (and in this case unprecedented) conditions will be permanent. LINK

Continued Peloton struggles: CEO Barry McCarthy stepped down this week after 2 years with the company. Peloton also laid off 400 people (roughly 15% of its workforce), which is the fifth reduction in force since the company’s peak of 8,600 staff in 2021. It has been a rocky road to restructure cost, remedy supply chain issues, and right-size demand forecasting since the pandemic. Shared are down 97% since the end of 2020 which has basically erased $43B in market value. I was hopeful the restructuring plan the executive team was executing against would help right the company, but it seems there are still some major issues here. Another layoff unfortunately isn’t the long term fix to the systemic issues they are clearly still facing. It’s truly a shame since their struggles seem to stem from operations vs product issues. LINK

Cloud provider growth & market share update: According to Synergy Research, cloud revenue grew 21% YoY in Q1 2024 to $76B. The US is still the largest cloud market, and grew 20% in Q1 2024 (and is larger than the entire APAC region). Another report from Altimeter also validated the rebound from what was a rather tepid 2022-2023. AWS is a $100B run rate business with roughly 31% market share. Microsoft Azure is a $76B run rate business with roughly 25% market share. Google comes in with roughly 11% market share. AI has driven a renewed interest in what has been a relatively mature industry for the last couple of years. There is still some significant growth in the tier-1 and 2 cloud provider market even with the 3 major players continuing to own 2/3 of the market. While I expect the US to continue to have the largest scale for some time, there is a lot of growth potential in Asia in particular. SYNERGY; ALTIMETER

Asia Pacific data center growth: Current M&A deal volume in 2024 has reached $840.47M which is 50%+ of the global total so far. Analysts expect 2024 to surpass the $3.45B record high 2023 deal volume. Another data point that validates that a) data center spending is incredibly sticky and b) AI is driving an insatiable hunger for quality data center capacity. This reporting comes off of the heels of news last week that Microsoft was investing $2.2B in Malaysia on top of the company’s previously announced $1.7B investment in Indonesia. LINK; LINK

TikTok music deal: After quite a bit of posturing, Universal Music Group (UMG) and TikTok reached agreement to license music back on the social media platform. Interestingly, Taylor Swift allowed her music back on the platform last month ahead of this deal being announced in a somewhat puzzling move given her support of smaller artists. Ultimately there are clearly benefits to being on TikTok, especially as a larger artist, though unclear how successful smaller artists will be even with this new deal. LINK

Earnings

Meta (NASDAQ: META):

• 1Q revenue up 27% YoY to $28.65B. Net income more than doubled from a year earlier to $12.37B. EPS landed at $4.71 beating analyst consensus.

• Despite an overall great earnings performance, the stock initially dropped more then 10% on weak 2Q guidance and increasing capital expenditures. It turns out there is some limit to the amount you can spend on AI.

• Threads now has move then 150M MAUs, up from 130M in February 2024. Impressive growth, especially in just a few months. While there is still a long way to go to get to the billion user level, does provide another advertising opportunity. Business Insider reported that Threads now has more DAUs in the U.S. than X.

• LINK

IBM (NYSE: IBM):

• 1Q revenue up 1% YoY to $14.46B vs $14.53B consensus; software revenue up 5.5% YoY to $5.9B, and net income of $1.6M (up from $927M YoY).

• Despite the beat on EPS, the miss on revenue and overall slower start to 2024 initially sent the stock down (-5%), well below the stock’s last 21-day and 50-day moving averages.

• Overall positive sentiments from most analysts on the AI strategy, but tempering of expectations for 2024 despite IBM reaffirming full-year guidance.

• LINK

Amazon (NASDAQ: AMZN):

• Revenue up 13% YoY to $143.3B; amazing growth in net income ($10.4B vs. $3.2B YoY) and operating income ($15.3B vs $4.8B YoY).

• AWS continues to see enormous growth and is not a $100B+ annualized business (up from $90.4B in 2023); growth to 17% (vs 12% YoY).

• Also of note is the ads business generated $11.8B in revenue last quarter with 24% YoY growth. For comparison, YouTube’s total ad revenue over the same time period was $8.1B.

• LINK

SK Hynix (KRX: 000660):

• Revenue of $9B which more than doubled YoY. Beat analyst expectations for operating income ($2B) and the largest quarterly in 2 years.

• Last week, the company announced new investment with TSMC to develop new next-gen, high-bandwidth memory for Nvidia (HBM4) to deliver at scale in 2026.

• SK Hynix is the #2 memory manufacturer in the world behind Samsung; combined the two companies have 70%+ of the global market. Memory is critical to AI workloads in particular (in reality, all computing), and there are really only a few significant players in the market that can deliver the high performance memory needs. A lot of focus has been on the core CPU and GPU fabs, but memory is something to watch.

• LINK

AI

NIST GenAI: NIST announced a new program to evaluate Generative AI with focus on many of the challenges of this class of technology. Key initiatives focus on text and image “content authenticity” detection systems and support software development of guardrails and detection tools. This is one of the first major actions from NIST around the Biden AI executive order. I am cautiously optimistic as there are clearly a lot of issues with deepfakes, misleading AI generated content, etc., but unfortunately it is unclear how much funding this new unit will have to drive real impact. Many of the announced initiatives are pilots right now with results published as soon as February 2025. LINK; PRESS RELEASE

NYT profile on Hassibis and Suleyman: Profile of Demis Hassibis (Google DeepMind) and Dr. Mustafa Suleyman (Microsoft AI) who both grew up in London and are now leading some of the most important AI organizations in the world. LINK

Snowflake Arctic: The company announced a new LLM optimized for tasks such as SQL generation, coding, etc. under an Apache 2.0 license. Open sourcing the model is a further sign that the LLM itself isn’t really the differentiator in the market these days, but the tools to build, run, and manage models on your platform are. LINK, HUGGING FACE

Apple OpenELM: Apple announced a new family of LLMs with 270M-3B parameters designed to run on-device and pre-trained and fine-tuned on public datasets. Apple is finally starting to bring some more focus to leveraging the power of the in-house silicon to support on-device AI, but I presume are likely running into limitations on RAM for larger LLMs, especially on the lower end device configurations. Truly an age-old computing constraint- you can truly never have too much RAM. LINK

Meta’s Llama 3: Mark Zuckerberg shared his company’s latest LLM was downloaded over 1.2M times in the first week. LINK

Bain GenAI survey: Interesting survey on how and where companies are deploying Generative AI. Overall most use is focused on short-term cost savings and small improvements to products and services. Significant, transformative use cases are less common. Lots of focus on build vs buy as well. LINK

M&A & Venture

IBM acquires HashiCorp: IBM announced the acquisition of the cloud infrastructure management company for $6.4B with the target close by EOY 2024. There were rumors HashiCorp was considering a sale earlier this year. Hashicorp has not won too many fans in the engineering or partner community after changing the Terraform license model from open source license to Business Source License model in 2023. The company perviously went public via IPO in 2021, and unfortunately has faced questions of how to expand revenue growth since- another case study in how hard it is to build a business based on open source. IBM is hoping to boost their cloud infrastructure product portfolio in addition to the Red Hat unit. IBM stated HashiCorp will remain a standalone subsidiary after the transaction closes later this year. ANNOUNCEMENT; WSJ

Nvidia acquires Run:ai: The AI infrastructure and orchestration service company is being acquired for between $680M-720M. Run:ai had raised $118M in funding with the latest round ($75M Series C) in March 2022 valuing the company at $388M. Not entirely surprising since Nvidia and Run:ai have been working together since 2020, but does likely help focus resources and product roadmap on Nvidia’s DGX Cloud. Likely not as impactful of an acquisition for Nvidia as Mellanox, but does showcase the importance of software for the future of Nvidia. LINK

Augment: AI coding assistant startup raised a $277M Series B at $977M post-money valuation as it emerges from stealth. Investors include Eric Schmidt, Index, Sutter Hill, and Lightspeed. This follows the company’s $25M Series A led by Sutter Hill. Augment plans to roll out the tried and true SaaS GTM playbook. With the excitement and early successes of coding assistants, I would imagine they will see significant adoption. Their largest competition is Github Copilot right now, so I would presume Augment’s focus will be on SMBs to start. LINK

Silicon

Apple M4: Mark Gurman published a report on the upcoming Apple silicon, likely to debut in the new iPad Pro set to be announced at the March 7 event and make it into the new iMacs, MacBook Pros, and Mac minis later this year. Apple is on a breakneck pace in delivering new M-series chips. M1 shipped in 2020, M2 shipped in January 2023, and the M3 shipped in November 2023 (~10 months later!). I’m guessing that the M-series release schedule will ultimately start to ease into the annual A-series timing as soon as this year with a (hopefully) full refresh of the device lineup to follow. The byline here is that Apple is doubling down on on-device AI features with this generation of silicon in particular, though more to be told if these features are for all M-series or simply the M4 and successors. LINK

TSMC A16 process: The company announced a new chip manufacturing technology called A16 with plans to start producing 1.6nm chips by 2026. While Intel was the first to announce backside power (new approach to deliver power more efficiently to the chip) with their new 14A (1.4nm) process earlier this year, TSMC will likely come to market faster with this new process. TSMC owns roughly 60% market share for foundry services (for reference, Samsung comes in next at 13% market share). LINK

Qualcomm Snapdragon: The company announced details around the Snapdragon X Plus and X Elite chips. While initial benchmarks suggested similar performance to Apple M3, AMD Ryzen 9, and Intel Core Ultra 9, multiple sources are claiming that these benchmarks are likely not achievable especially with the claimed settings used. Qualcomm continues to be in catch-up mode. LINK

Legal

NY State AI Committee: The New York State Court system announced a new advisory committee around AI and the use of the technology by the courts themselves. While there is certainly a lot of promise here, there are significant risks, especially around the ethical implications, for some use cases. LINK

AI litigation merry-go-round: This will continue to be a topic to watch, but there was a new volley of motions to dismiss, reply briefs, and opposition. We really do not have, and likely won’t for some time, any resolution to these AI cases to give us direction within the U.S.. I expect more dismissals until we have more regulation.

• Huckabee v. Bloomberg: Plaintiff filed opposition to the motion to dismiss. LINK

• Intercept Media v. OpenAI: Both Microsoft and OpenAI filed motions to dismiss. LINK, LINK

• Anderson v. Stability AI: Everyone submitted their reply briefs in support of the motion to dismiss. MIDJOURNEY, STABILITY AI, DEVIANTART, RUNWAYAI

Other news

Meta Oversight Board layoffs: Reports of pending layoffs at Meta's Oversight Board is a bit troubling, especially in an election year. Most people don’t realize this independent board exists, and is trying to bring more accountability to the Meta platforms. Unfortunately there has been a lot of criticism around how quickly they move to address issues like AI, censorship, etc. They have also had challenges raising funding outside of Meta to support their operations. Originally funded by Meta with a $130M grant in 2018 with another $120M grant in 2022. LINK

China State Secret Laws: New revisions to the State Secret laws requiring companies to delete leaked information and comply with investigations have been published. Tencent, Weibo, Douin, and other internet companies were likely the inspiration for these updates. More censorship and more spying. LINK

Data Center development backlash: Datacenter be built across the country, but some states are starting to introduce legislation to curtail the industry. Northern Virgina, Georgia, Arizona, Illinois, and Arkansas have either passed laws or restricted development. Concerns around energy use, water use, proximity to neighborhoods, and tax revenue. States clearly see the opportunity, but are trying to make sure continued development is both economically and politically worthwhile. LINK

Subscribe now