Lenovo AI Now - Device-based Personal AI on PC
Device-based AI software integrated into PC. Build a personal knowledge library and solve PC usage problems. Provide a local, safe, private AI-using environment.
Client
Lenovo
Lead UX Design in the team, and responsible for the project management; Individually contribute to 80% of interfaces and function models design; Direct report design solutions to client global SVP weekly.
Key Contribution
1 Product Manager, 2 UX Designers, 1 Visual Designer, 1 User Researcher, 1 UX Writer. (6 PMs, 3 UI Designers, and 20+ Devs from the client side)
Team Scale
05.2024-09.2024
The product was launched on Oct 15th, 2024, (View News) and will be used as a build-in software for all new PC consumers and SMB users soon globally.
Time Range
All the content and visual layout on this page were reorganized after the project individually.
“Our AI is a bit dumb...”
It all started from the criticism by the Global SVP of the world’s largest PC manufacturer ignited a pivotal challenge. As AI surged in 2023 and 2024, Lenovo set out to develop its own Large Language Model (LLM) with the vision of "AI for All"—a smart, private, device-based AI that works without internet connectivity.
After launching a version in China, they planned a global rollout, but both faced harsh feedback like "stupid" and "the dumbest AI." The issues stemmed from flaws in the LLM, leading the CEO to set an urgent deadline for improvement by October 2024. Our UX consulting team was brought in to deliver a user experience transformation, giving the AI product a much-needed "makeup", rather than an overall Genetic Revolution which includes the LLM base.
Clear distinction between PRC and ROW versions: The design needed to create a clear differentiation between the Chinese (PRC) and global (ROW) versions, covering both interaction flows and visual style.
Challenge
Balancing user-friendly and tech-friendly design: The challenge was to introduce a design that catered to user needs while working within the limitations of the LLM, striking the right balance between usability and technical constraints.
Tight deadlines: The design process and development faced significant time pressure, with a product demo scheduled for IFA Berlin in September 2024. The MVP’s experience “makeover” needed to be completed by the end of August.
Complex stakeholder dynamics: The client’s urgency created ambiguity in the roles of internal stakeholders early in the project. With a team consisting of six PMs and three UI designers, decision-making was often delayed due to hesitation and siloed responsibilities.
Results
A usability test with 30 English speakers resulted in an overall experience score of 4.4/5. All feedback and iteration points were addressed during the design phase, ensuring a polished user experience.
During the final presentation, the Global SVP commented, “This is exactly how it should look and interact,” validating the design approach. We now eagerly await real consumer feedback once the product hits the market.
After leading the Thoughtworks design team to complete the MVP requirements, Lenovo’s internal team continues to build on our foundation. The product is set to launch by the end of 2024 as pre-installed software on new PCs across the US, Europe, Australia, and other English-speaking markets.
My Contribution
Led the experience insights analysis based on the PRC version to inform design decisions for the global product.
Facilitated co-creation workshops with the client team, driving creative collaboration in feature solution & visual brainstorming.
Acted as the UX Designer for 80% of the MVP, shaping the core user experience.
Participant in usability testing. Facilitated 15 test interviews with global candidates.
Overall project management, including recruiting key team members (visual designer and English UX writer) and overseeing usability testing candidate recruitment.
The Client Status
The Client Team Structure
At the beginning of 2024, Lenovo launched its AI product in the Chinese market, Lenovo Xiaotian (联想小天). After observing its performance and market potential, they decided to expand globally, using their proprietary large language model (LLM) adapted for English. Initially, the plan was to reuse the existing design, only swapping the language model. However, after a review by the company’s Global SVP, internal teams recognized usability issues and brought in an external consulting team (our team) to address the challenges.
The product’s key features include local processing, privacy, and a personal knowledge base. Using a conversational AI interface, it aims to help users solve everyday PC-related issues and improve productivity in personal and work-related tasks.
Our Team Strategy
Thoughtworks Project General Schedule
Before the project started, Thoughtworkers identified internal usability issues through its business partnership relationship. Then the team conducted an preliminary experience review and evaluation.
Once we were onboarded, I took the lead in driving the overall project, also managing project coordination. Our team worked collaboratively to analyze the pain points identified in the preliminary review, categorize the issues based on internal testing results, and conduct an industry research to assess the strengths and weaknesses of similar products. Through co-creation workshops, we developed solutions to address the root causes of these issues and set the direction for the product’s visual style.
2 designers (including me) were responsible for different product modules, with key interfaces in the main user flow handed over to the visual designer for further refinement. We also conducted usability testing. Additionally, recognizing the lack of an international perspective within the client’s execution team, we engaged a Thoughtworks North American UX writer to collaborate on refining the software’s design for global markets.
Preliminary Experience Diagnose
2. Experience Review Method:
This module was completed by 2 Thoughtworks Experience Designers (including me) and belongs to the pre-sales stage.
My main responsibility in this stage was to define the experience walkthrough scenarios and analyze the details of the framework and experience reports in each scenario.
1. PRC Ver. Product Status (before May 2024):
The PRC version of the AI product had significant design issues.
During its initial development, the team didn’t clearly define how users would interact with the product. As a result, it was designed as a narrow sidebar fixed to the right side of the screen, expanding into a large window on the left when accessing the personal knowledge base. Additionally, the product’s visual language and interaction style felt more like an engineering tool rather than a user-friendly, consumer-oriented utility.
3. Preliminary Experience Diagnose Results:
- First Look Fails to Convey Core Value
Positioned as a professional personal assistant, the product’s homepage lacks the layout and messaging to effectively communicate its key strengths in boosting productivity.
- Poor Information Architecture Limits Exploration of Key Features
The product’s structure doesn’t encourage users to explore core capabilities like the “personal knowledge base” and “local AI.”
- Inconsistent Design Language and Visuals
The icons and UI style are mismatched, and the overall visual design fails to represent the product’s “personal intelligent assistant” identity.
- Unclear and Rigid Copywriting/UX Writing
Copy on the homepage and knowledge base setup page lacks clarity, making it harder for users to understand and navigate the product.
- Multiple Usability Issues
There are ongoing usability problems with login, button feedback, upload status, and knowledge base management, impacting the user experience.
Brief of Desktop Research (till Jun. 2024)
1. What are industry players doing with AI?
AI product development in the smart hardware and software industry focuses on empowering developers, integrating third-party applications, and customizing cloud-based models. By enabling developers to tap into cloud or local AI capabilities, the development process becomes more efficient and accessible. Partnering with third-party applications enriches the product ecosystem, while collaborations with external products eventually lead to in-house model development.
Smartphone manufacturers are creating versatile AI solutions that adapt to multiple modes and scenarios. These include AI applications for writing, calling, translation, office work, photography, and personal assistance, offering a range of functionalities beyond a single use case.
Office IM platforms, like DingTalk and Slack, focus on integrating work information flows to provide AI-powered “secretary” functionalities. These solutions streamline workplace communication by summarizing information, highlighting key points, and boosting productivity.
Content creation tools leverage AI to overcome creative bottlenecks, offering content generation capabilities. These tools assist users in brainstorming, inspiring creativity, and producing content directly.
2. How does other provide the solution on Personal Knowledge Management?
Obsidian
SparkDesk by iFLYTEK
All stakeholders (department lead & AI product managers) and UI Designers participated in the co-creation workshop. 4 main activities were prepared, including functional expectation analysis (futuristic solutions), pain-point roast (realistic problems), practical solution analysis (criticism solutions), and visual moodboard co-creation.
After conducting 4 rounds of focus group interviews with the product team, design and development team, department leads, and the Global SVP, we identified key experience issues and expectations. We then analyzed and grouped them into 7 distinct areas.
Base on the elements of user experience design, design directions in product strategy, scope, structure, skeleton and surface are delivered to the Lenovo team.
The results was used as inspiration material in further co-creation workshop to brainstorm more grounded specific solutions.
Key insights & Similar Solutions by other product - Experience Walkthrough Report
Local personal knowledge management tool such as Obsidian is based on local files and encrypted data. It shows the bidirectional links, diagram views, and plug-in extensions enable users to organize and connect information efficiently.
Well: Build up the connection between information; Make sure the order of the information; Data search, etc.
Less Well: Multimodal information collection; Stimulate creativity, etc.
AI-empowered Knowledge Management Solution Example:
SparkDesk by iFLYTEK is a cognitive LLM in Chinese. The document parsing and understanding capabilities, making personal domain documents and structured data construct into retrievable knowledge.
Well: Ease of use and understanding.
Less Well: Build the trust by seeing the retrieving results; Lack of detailed description from the answer.
Internal Research & Co-creation
1. Root Cause & Design Insights Analysis
2. Co-creation Workshop
From the product design perspective, the MVP function priority chart is a key outcome after the workshop. The afterwards product design will firstly focus on the high-priority features.
AI Solutions by other Industry Players
Local Personal Knowledge Management Solution Example:
Visual Design
The visual design was the final activity during the Co-creation workshop. Visual keywords were pre-defined in the Lenovo team. We used these words to seek for further aligned description by other adjectives and pictures to create visual moodboard.
I was responsible for preparing the workshop materials, and visual design direction analysis in this section. The detailed final visual design was finished by a Visual Designer from Thoughtworks team.
Product Forms & Evoking - Default Window & Quick Accesses
Insights from Research:
Product Design Solutions:
4. Interaction Cards: Given the product’s PC-based features, design response cards from the AI PC assistant to enable quick control over computer functions.
Flexible product form, easy to be evoked and hide
Easy to use on specific functions, combine the usage scenario and the task-flow. Seamless experience.
Insights from Research:
1. Input Box Interaction: Consider input field interactions across various scenarios, including maximum text height, character limits, PE (predictive entry) calls, and additional file attachment options for queries.
2. Standard Text Cards: Standard text cards are the most common response format in AI Q&A interactions. Consider typical card interactions to design multi-state scenarios for text card interactions.
3. Information Cards: Identify model capabilities, client expectations, and feature priorities to design response cards that display various types of informational content.
Straightforward page layout. Quick file attachment interaction to ask questions directly.
Similar conversation flow to build up familiarity and reduce learning costs.
Evidence-based responses. Answer related files will present with the responses to give user trust and easy to recall personal files.
Knowledge Management:
Import files from local device, cloud drive, or mailbox.
Rapid diagnosis and analysis of all files, sync file changes to AI Now, and import files.
To specific file, the user can edit file labels, recall file originally located, and delete from Knowledge Library.
Select multiple files to achieve multi-management.
OOBE & On-boarding Guidance
Insights from Research:
Flexible guidance types. Sort out core features and scenarios, and design matched guidances.
Short & straightforward information details explanations/tips/feedback. Education the AI limitations to the users.
Design Principle:
Expectations from the Lenovo Team:
Team is consulting with Microsoft Team to ask for collaboration and integration within Microsoft Office softwares to support quick access inside.
Team wish to breakthrough the design from the PRC team. Clear differences are needed.
AI Conversation Flow
Design Principle:
Smart：Clear and instant conversation feedback
Rigorous: Questions & Answers references
Humble: Collect user’s satisfaction or recommended next step actions
Knowledgeable: Various forms of dialogue cards that suitable for different scenarios
Product Design Solutions:
Knowledge Assistant
Knowledge Library & Conversation:
Mini-window
Other Pages
Product Design Solution - Software OOBE:
Product Design Solution - Page OOBE & Tooltips:
Standard Window Mode: The core feature of AI Now, allowing users to interact with software via conversational AI.
Quick Access Window Mode: A convenient feature of AI Now, activated by shortcut or voice command, for easy AI access in multitasking scenario.
Integrated Software Ecosystem: AI Now integrates with common office tools, providing an assistant with features like right-click options to polish documents or add files to the knowledge base.
To avoid importing mistakes that limited by the Lenovo LLM, showing files that currently in the importing queue, and failed importing.
Support single file pause, recall originally location, and delete, also easy for multi-management.
While edit label from single file, the user can directly create or delete in the row.
Select multiple files and click Add Label to add it in batches in a pop-up window.
Quick file search supported. Left file library area changes into Knowledge Found. Click on AI response anytime to review the previous file search requirement.
For Email importing, the user can search for specific emails with keywords/sender/receiver, time range, and also attachment.
After selecting emails to be imported, AI Now supports only import attachment or the overall email.
As mentioned in the previous section, the mini window is a quick access for the AI Now (Lenovo IQ). It only support convenient features, and navigate to the default window when needed further actions.
Floating button for users who love it always on the screen. Turn transparent when lose focus for 10 seconds. Able to be closed in the Setting.
Default input box while clicking on the floating button or active by pressing shortcut or voice command.
Support general quick local AI chatting. Top-right button to go to default large window or close the chat.
Support quick knowledge library file search. Find files/knowledge needed and navigate back to the original location or start a conversation with the file attached.
PC setting general, like brightness adjustment, mode selection/switch, basic parameter checking.
Usability Test - Preparation
Why (should we take the test): Majority of the product features are defined by the Lenovo & Thoughtworks internal teams, and lack of real users insights - due to the fact that the product form is new, and no direct competitor product in the market.
What (type of things should be test): Key interfaces to present the major product use cases.
Who (are the facilitators and participants): 2 Thoughtworks UX Designer recruiting English native speakers who mainly living in NA, and European who use English as working/daily language. Mainly cover PC experts, Day-to-day users, Unskilled users with DEI consideration.
Where (are we going to take the test): Online test.
When (is the test take place and duration): After finishing the key interfaces UI design. Planning to take the facilitator turns for testing maximum 4 people one day, overall 2 weeks.
How (should the participant engaged in the test): Zoom meeting with clickable prototype sharing. Facilitator introduces overall product background and let the participant experience the prototype. The test needs participant describe loudly what information they received from the page, and facilitator will ask questions after each page experienced.
How many (participants will be recruited): 20 from general public, 10 from Lenovo internal.
Usability Test - Testing
A Glance of AI Now User Testing Report
Through internal discussions and early diagnostic insights, we identified areas needing validation, such as assistant switching within conversation flows, understanding and usage of the personal knowledge library, comprehension of the explore page, and usage of the quick-access window. To address these, we outlined specific questions for each module to validate design comprehension and interaction methods.
We asked the participants take a general assessment after all the interview tasks. The participants gave scores on Interaction, Vision, Operation, Function, Overall Satisfaction, and gave negative/should-be-improved ideas on specific items.
Usability Test - Results
Overall satisfaction score 4.41/5.0
A 50 pages testing report was delivered after the test, introduces the testing process, results, key findings, further improvements with priorities (based on criteria from urgent to considerable: impact basic usage, clarity, understanding, affect development, experience enhancement, etc.)
The key results about design iteration during the project will be introduced in the next chapter: Iterate.
Key Design Iterations During the Project
1. Simplified prompt example cards:
2. Optimized mini-window use cases:
3. Completed user guidances in the product:
UX Writing
The prompt example (or prompt engineering-PE) cards appear in conversation pages to show LLM basic ability, and will disappear once the user click on the input box.
Previously, Lenovo team wish to sort out the PEs and show as much as possible to give user ideas about what to do with AI assistant.
After the design of OOBE/user guidance and usability tests, we simplified the PEs and set it automatically change in several seconds.
In the initial design by the TW team, the quick-access window was intended for rapid use, offering features like quick function calls based on user behavior, knowledge searches, and temporary AI Q&A. However, Lenovo internal feedback suggested adding assistant switching and access to all PC cards, which contradicted the original separation of functionalities between the default window and mini window.
After conducting the testing and further discussion with the team, we removed the assistant-switching feature and simplified the PC cards to support only commonly used ones. We also streamlined the quick-access window by removing the function call feature.
In the Design chapter - OOBE section, I defined the guidance hierarchy as Software OOBE, Page OOBE, and Active-displayed Tooltips.
Hover tooltips was mentioned a lot in the usability testing. We improved the hover tooltips and made an overall list of them in the product. This will significantly improve the integrity of the product and enhance the real scenario experience of using it. Furthermore, closed the loop of OOBE/Guidance design.
Other Iterations During the Project
Based on both internal team input and user feedback, we paid close attention to product language and wording. To enhance clarity, a native English UX writer from TW North America was brought onto the project to work with me on refining the language.
Lenovo Global Design Team Review
Following an Global Design Review, we provided visually focused optimization suggestions for Windows app configuration limitations and interactions, such as resizing icons and adjusting settings button placement—without altering major experience modules. Product naming and logo design discussions are still ongoing. (till Sep. 2024)
Future Version Exploration
Current version of design focus on experience improvement. However, we presented a Genetic Revolution Version ideas for the product future development with the power up of LLM, including a smarter knowledge library for team users at work, multi-media file search on specific frame in a video, etc.
Designing an AI Product
As the project was wrapping up, another leading smart device company approached us, eager to learn about AI-focused design principles. After my showcase presentation, their questions quickly turned to LLM capabilities, almost pressing me to explain key distinctions between general product design and AI product design: What are the five elements of AI product design? What should a team without AI experience do to prepare? What started as a project sharing became more of an impromptu interview. It soon became clear that they were gearing up to build their own LLM, hoping my team could outline a complete AI design methodology for them.
Many digital and smart device companies are racing to integrate AI, attempting to revive legacy products. But what if their initial ambitions misalign with real user value? Making profits in business objective should prioritize the user values, this stays the same for designing an AI product. AI is a powerful tool, but like any new technology, it should be approached thoughtfully. My viewpoint is that AI is a form of technology - just like facing a new material in designing a wearable physical product - the criteria while designing various due to specific techs, but the methodology logic stays the same (or similar, because of using design thinking).
Team Leading & Coordinating
Leading a project from start to finish for the first time, I found the two biggest challenges were coordinating across international stakeholders and managing team responsibilities.
With Lenovo’s large, multi-level team, we conducted interviews with global SVP, department heads, and project executors. This helped us understand varied perspectives within the company, align goals, and assess each person’s influence on decisions—essential for managing requirements and resolving questions. Additionally, based in China for this international project, I took on the task of overseeing communication with global teams, often spending extra time to ensure smooth progress at each step.
Within the Thoughtworks team, initial contract confirmations took longer than expected, so I set up daily task tracking and clearly aligned staffing according to each member’s skills and growth interests. This proactive approach extended to coordinating with our North America Thoughtworks team, such as in recruiting a UX writer, to maintain consistent and positive momentum across the project.