Pebble Index 01 - Webhook Feature Requests

Pebble Index
1

Pebble Index - Webhook Feature Requests

Hey there, first time posting on this Pebble forum :slight_smile:

tl;dr: Pebble team, can we please get more customization for the Index Webhooks (in my case to help mitigate the failing in-app transcriptions)? E.g. call the webhook even if the in-app transcription failed, retry the webhook if it failed, etc.

Context & issues with local-only transcription

I’ve been playing with my Pebble Index 01 Ring since I received it a week ago, and had a pretty bad experience with local-only transcriptions.

Indeed, most (80-90%) of my ring activations end up in “Unable to transcribe Index recording” errors. I’m discussing that with support already.

For what it’s worth, I configured it to favor French even if Parakeet is able to do multi-languages with detection natively.

Alternative transcriptions

Seeing that it was using parakeet-tdt-0.6b-v3 as the local model, I started comparing with other pieces of software that use this model.
My approach was to extract the MP4 files using the Pebble app export feature, and get them to be transcribed with other tools.

First with Murmure that I contributed to and which provides a local API, and then with Audio.cpp (can’t link to these as new users are restricted to 2 links in a post).

Both Murmure and Audio.cpp with Parakeet worked better than the Pebble in-app transcription, without any specific settings. I have no clue where the difference could come from and I’m not knowledgeable enough about ASR/SST models workflows and direct usage.
This is likely a hint for the Pebble team to look into: the Parakeet usage in the app can maybe be improved to get better and more consistent results by looking into how Murmure and Audio.cpp use the model?

Automating alternative transcriptions

From that point, I figured that I could mitigate the Pebble inconsistent/failing transcriptions by doing my own thanks to the webhook usage.

I’d like to note that I very much appreciate the sheer existence of a webhook call possibility - this is one of the reasons that convinced me to get the Pebble Index! I love that it makes it easy to go “oh, the app doesn’t do something I’d like it to do? Let’s just hook from it and implement it on my own”.

I implemented a pretty simple webserver (available at GitHub - Horgix/recueil: Recueil — A Pebble Index 01 ring webhook receiver (https://repebble.com/index) · GitHub for those interested; written in Rust) that:

  1. Receives the audio and optional transcription through the webhook
  2. Stores the audio locally on the server
  3. Stores the history of received recordings along with all meaningful information
  4. Calls Audio.cpp to transcribe what failed with the Pebble app (or even transcribe everything to more accurately compare the Pebble / non-Pebble results)

However, in the end I’m not able to actually mitigate the failing transcription with that workflow, and it comes down to how the Pebble app handle webhooks calls.

How the Pebble webhook currently works

From my trial & error, here is how the Pebble app behaves when it comes to webhooks:

  • The webhook is only called once the transcription has been successfully done in the app. That means that:
    • If the transcription fails, you retry it, and it succeeds (e.g. by switching from “Local only” to “Cloud”), the webhook is called at that time
    • The webhook is never called unless a transcription succeeds, and especially even if is configured to send Recording only. This is a bummer and what makes it irrelevant for my usecase as-is
  • If the webhook call fails, it is never retried and there’s no way to get it to trigger again

I believe that the “How webhooks work in the Pebble app” was already partly discussed in Index Webhook implementation questions

Feature requests

Here are the features I would love to see land in the Pebble Android app when it comes to webhooks (ranked by my own opinion of how important it is for my personal usage, for what it’s worth):

  1. Call webhook even if transcription fails when the webhook is configured to send Recording only (transcription result has no way to matter)
  2. Have an option to enable calling webhook even if transcription fails, when sending Recording + Transcription
  3. Provide visibility into webhook calling and result. For example, when viewing a recording in the app, in a similar fashion to the “Done” item below the transcript, have a “Webhook called successfully / Failed to call webhook” item
  4. Provide a way to retry the webhook call - even if it’s a manual “Retry” button akin to the transcription retry one and not automated at first

Wrapping up

I believe the webhook option is an awesome feature to have, but that it would be drastically better with only a few improvements. I’ll be super happy if the Pebble team considers the features suggested above and can’t wait to build more things out of it :smiley:

2

Some of this is already in progress and I believe we plan to do all of this. As for the transcription issues, we’re working on fixing it and the bug report is appreciated. Preferably you wouldn’t need to work around it and the local model should work completely offline!

You can always use remote transcriptions as a workaround for now if it doesn’t work at all for you, these are still kept entirely private unless shared with us via a bug report.