Customers send voice notes and photos, not text. Does the AI understand?
Short answer: Yes — the AI in Merpati listens to voice notes, reads images (receipt screenshots, product photos, posters) and PDFs. It converts them all to text, then answers from the business information as usual. When a voice note is unclear — noisy, cut off, two words — the AI does not guess: it asks the customer to type, and flags the conversation for the owner.
Why does this matter for Malaysian businesses?
"Yall stressing me. Awak hantar apa? Hi ke? Salam? Nak order bunga? Ahhhhh"
— @azhar.nasir, TikTok, 29 July 2026 (8.1k views)
[…What did you send? A "hi"? A greeting? Do you want to order flowers? Ahhhhh]
Malaysian customers do not type. "Sis, the one I saw in the live, dusty pink, is size M still available, I want two, ship to Kuantan" — a 40-second voice note. Or a product screenshot from Instagram captioned "got this?". An AI that only understands text goes silent on that message — and silence on a customer's message means the customer leaves.
What exactly can the AI understand?
| Type | Example | What the AI does |
|---|---|---|
| Voice note | A 40-second order, regional accent, mixed Malay-English | Converted to text, answered from the business information |
| Image | Receipt screenshot "paid", product photo "got this one?", a poster | Reads the text in the image and understands what is being shown |
| Forms, quotations, documents | Read and answered | |
| GIF / short video | A product clip | One or two frames are taken to understand the context |
How accurately are voice notes transcribed?
Merpati's internal test, 28 July 2026: 20 Malay audio samples, 19 transcribed accurately. The one failure was a synthetic test voice whose pronunciation was genuinely broken, not a human voice. Traps found and fixed in that test: "hari Jumaat" (Friday) once became "Harry, you mate", "Kuching" became "kek kucing" (cat cake). There is now a list of Malaysian words — days, cities, trading terms — plus the owner's own product names, guiding the AI to recognise what it hears.
What happens when a voice note is unclear?
An AI that "understands" a voice note it did not actually understand is more dangerous than one that does not understand at all. If the transcript is empty, under three words, or has unclear parts — the AI does not answer confidently. It replies: "Sorry, that voice note wasn't very clear — could you type it out, or shall I connect you to the admin?" The conversation goes into the "needs you" list in the inbox, and the owner listens to it themselves.
Does the AI send images back?
Yes. If images are attached to a product in the business information, and a customer asks "do you have a photo?" or "which colours are there?" — the AI sends the image along with the answer. A maximum of two images at a time, so it never becomes spam.
Are there limits?
Yes. A voice note that is too long or a file that is too large is declined with a polite message to the customer (not silence), and the owner can see in the inbox what was declined. No media is stored — the AI reads it, converts it to text, and discards the file. What remains is the text in the inbox.
Test it with your own accent
Sign up at merpati.my, connect your number, and send a 30-second voice note from another phone. See whether it understands, or asks you to type.
Sources
- TikTok video by @azhar.nasir, 29 July 2026 — Apify data 2 September 2026, quoted verbatim.
- Merpati internal test, 28 July 2026 — 20 Malay audio samples, 19/20 accurate after the Malaysian word list was added; 4/4 images, PDF and video frames. Test records kept in the internal repository.