Transcripciones de audio/Video a texto usando ML & AI (y sin escribir código)

Posted on April 8, 2021April 8, 2021Categories Blog   Leave a comment on Transcripciones de audio/Video a texto usando ML & AI (y sin escribir código)

El otro día me plantearon un caso de uso que me pareció interesante explorar, y debido a que los resultados me fascinaron quiero compartirlo con Uds.

En esta época de anuncios acerca de “…apriete un botón en su computadora y un modelo de Machine Learning o Inteligencia Artificial hará todo lo que Uds. Quiera…” al CMO de un cliente nuestro le pareció súper natural el planteo del siguiente caso:

Transcribir a archivos de texto horas y horas de grabaciones del sistema de atención a clientes.

Ok my friend, desafío aceptado.

Los insumos

Íbamos a utilizar los clips que nuestro amigo menciona, pero como no podíamos acceder todavía a sus archivos, preparé una serie de clips de audio para entrenar los modelos y poder presentar una prueba de concepto completa a nuestro cliente.

Para esto utilicé porciones de video que extraje de algunos webinars que he dictado a través del tiempo y que tenía en el disco de mi laptop. Algunos en español, otros en inglés.

Para el ejemplo que describiré ahora, pueden ver el video completo acá: https://www.decisionscience.bi/2020/09/20/demistifying-price-elasticity/

Video.png

Subí el video a AWS (específicamente a un bucket de S3), para agilizar el proceso y tenerlo siempre disponible.

Armado del Modelo

Con una simple interface drag and drop, podemos definir el flujo de procesamiento del modelo.

flow.png

En nuestro primer nodo, debemos definir los datos que utilizaremos, lo que se hace fácilmente con un simple JSON cuya estructura la podemos tomar de la documentación disponible:

json.png

En nuestro caso el Id del video, el email a notificar cuando el job se complete y la url conteniendo el activo de datos a utilizar (en este caso nuestro video).

Asignamos las variables definidas en nuestro JSON al payload del modelo, seleccionamos el motor de procesamiento y capturamos los errores (en caso de que surgiera alguno).

Salvamos, hacemos deploy y ya nuestro modelo está listo para utilizar.

Así que pasamos a Alteryx.

Para ejecutar nuestros modelos, existen dos etapas:

  • La primera es disparar el job de ejecución al servidor
  • La segunda es leer los resultados.

Para ello, tenemos en el Gallery dos herramientas (https://gallery.alteryx.com/#!app/Veritone-aiWARE-Tools/5ff7444e0462d71998d0c4be), una para cada proceso.

Paso 1: Lanzamos el job al servidor:

Run AI.png

Seleccionamos desde dónde estaremos alimentando los datos, en este caso desde S3, pero podríamos estar subiendo archivos desde nuestra máquina, tomar datos desde una herramienta conectada al conector de input, o seleccionar archivos subidos anteriormente.

Luego le damos un nombre a nuestro proyecto (en este caso “Webinar Transcription”) y como seleccioné que iba a utilizar un video alojado en S3, debo colocar las credenciales de acceso.

Por último, en la parte inferior de la herramienta, seleccionaremos qué modelo vamos a utilizar. Disponemos de la opción de flujos standard o custom (creados por nosotros, como el que les comenté más arriba)

flowOptions.png

Y eso es todo lo que necesitamos, así que a ejecutar el workflow.

flowExecution.png

Paso 2: La segunda parte del ejemplo involucra el acceso a los resultados del modelo de transcripción, donde esperamos leer la transcripción del contenido.

Para eso, en un flujo nuevo, vamos a utilizar la segunda herramienta mencionada más arriba (Query aiWARE Data):

QueryAIware.png

Esta vez, vamos a seleccionar de nuestras opciones “List of Assets”, que corresponde a los resultados de los modelos ejecutados por nosotros previamente.

En caso de que exista algún proceso nuestro, todavía ejecutándose en el servidor, lo veríamos debajo de la sección “Pending AI Jobs”.

Un poco de parsing a los resultados recibidos y podemos acceder a toda la transcripción del audio.

transcripcion.png

Todo el texto en un campo único, o

transcripcionWbyW.png

Palabra por palabra, con los datos en milisegundos donde aparece en el audio, y el % de confianza de su traducción.

Un caso de uso simple, pero de altísimo valor para las empresas. Pensemos en la aplicación de estas soluciones a transcripción de llamadas de Call Center, testimonios judiciales, generación de manuales en base a dictado, subtitulado de videos, en fin, sus aplicaciones son infinitas.

En breve les estaré compartiendo otros casos de uso (reconocimiento facial en videos, reconocimiento de logotipos en fotos y videos y otros más) en los que estoy trabajando.

Espero que les sea útil y no duden en contactarse si necesitan ayuda/apoyo.

Saludos

An Easy take on Artificial Intelligence (AI) for video and audio analysis

Posted on April 8, 2021April 8, 2021Categories Blog   Leave a comment on An Easy take on Artificial Intelligence (AI) for video and audio analysis

A couple weeks ago, I was presented with a solution that was in development, and I volunteered to be a beta tester. I always say YES!  to these opportunities, so I embarked in learning a new (yes, again!) platform to perform ML & AI. Read more about the Alteryx + Veritone partnership and the aiWARE tools.


So, why did I do it? Well, first I’m very curious by nature. Second, the demo I saw explained some ML & AI underworld to a basic guy like me. Finally, because it was fully integrated to Alteryx (so I thought I had that part covered). Very nice starting point to this journey.


The example I saw was a part of an Alteryx webinar where @MarqueeCrew  and @LeahK  (two very well known Alteryx superheroes) were talking, with the video switching faces as they talked. Four or five photos of each were uploaded to train the model (downloaded from Google Images).


Once the model was trained, it ran and voilà! The results show some very curious information:

  • It recognized the time where each identified speaker was talking
  • It recognized a speaker that was not trained (another Alteryx superhero @CharlieS
  • The model made the transcription of who said what at what time
  • What was the expression in the voice when they said it

For those with me so far, let me tell you… if your head is not exploding right now, it will be in a few paragraphs.

My First Test

So far, that was my experience with the demo, so I decided to do it myself (thinking I wasn’t capable of pulling this off myself). Big surprise! I received two Alteryx macros in advance: the aiWARE Input tool and the Run aiWARE tool (accessible on the Gallery).

As an Alteryx enthusiast, the first thing I did was to drop the Run aiWARE tool on to the canvas. I was pleasantly surprised that I was able to sign up for a free account from within the macro, so I did; yes, you can even do it from within the macro, without leaving Alteryx, and receive a free trial, too!

login.png
Signup.png

Now that I have my credentials, I was able to log in, and the main interface of the tool was presented:

RunAI.png

Select Data: Lists options to retrieve your data:

SelectData.png

Upload Type: If you’re going to be uploading a single file or a folder contents:

UploadType.png

Project Name: Just a name to keep your projects organized.

Local File: Since I chose to upload a local file.

Flow Options:

FlowOptions.png

Here starts the amazing journey for me. You can use standard workflows, published by Company, or you can use yours, through the Custom Flows Option.

For the Standard Flows, here is a partial list of those available to Alteryx right now:

SelectFlow.png

I decided to try the Facial Recognition first, based on two previous disappointing experiences I had with two other vendors and this kind of model. When I selected Facial Recognition, I was presented with a new drop down, Select Library:

SelectLibrary.png

And a very suggestive option to Train my First Library. Obviously I clicked the link.

In the same panel, I was presented with the Libraries section of the solution, where you can build, edit/update/expand or delete libraries for model training.

Library.png

I started to look into my work laptop and found some pictures of my kids, so I fed the library with them. The process is simple, easy and straightforward, with instructions at every step. It took me more time to find the pictures than training the model with them.

So finally, I selected the library for this test, and ran the workflow (I am a Browse tool guy in development, so I added one first).

RunWorkflow.png

The workflow throws a job to the server to start performing the Facial Recognitions. A very important thing to notice is there’s no need to wait until the task finishes, so meanwhile you can work on another things… (yes, I know, bummer, but your boss will love this!)

The execution results:

RunResults.png
RunDataResults.png

If you’ve been reading carefully, I said I received two tools and now is the turn of the second one to appear in a new canvas. The aiWARE Input tool asked me to log in again, and returned the following:

QueryResults.png

Selection Option: It allows you to choose from where you’ll get results.

SelectOption.png

List of Assets: In this case, all the jobs I previously ran.

ListOfAssets.png

Pending AI Jobs: Shows what jobs are still running on the server

ModelResults.png

I ran the second workflow (a pretty basic one, btw). 2.9 seconds for the results; I went directly to the results. It’s JSON, and I’m in Alteryx, parsing is no problem!

ModelOutput.png

JSON Parse to the rescue and after a little simple Alteryx magic, I have readable results:

JSONParse.png

I’m not going to enter in much detail on the results, because I think you need to test this and be amazed yourself, but I will comment on some of them:


Emotion detection (up to seven emotions tested by frame) and Confidence probabilities of each one:

FinalResults.png

Twenty-nine face landmarks evaluated per subject and their probability score:

29landmarks.png

Each person in the library recognized, based on a set of four pictures of each (the one that is blank is my goddaughter, whose picture where not included in the training set):

PersonsRecognized.png

Part of the video where they were recognized:

VideosEgmentRecognized.png

And a lot of other results, like if the model detected beards, sunglasses, poses, etc.

Conclusion? It took me 10 minutes from scratch to get the experiment done, starting with picture gathering, video uploading, model training and face recognition completed.

A more complex experiment

Since I was already hooked up with these cool new tools, I decided to get into a real business case. A couple of years ago, a potential customer asked us about feature recognition on shopping mall footage.


What a better way to test it right now!

I got a video online (a walk through the Dolphin mall in Miami that somebody posted on the internet), and decided to test the Logo Recognition Engine.

videoreel.png
videoFinal.png

I strongly recommend that you give this a test, I’m sure it’ll blow your mind as it did to me. If you have any questions, please reply to this article. Read more about the Alteryx + Veritone partnership and the aiWARE tools.

Machine Learning en el sector financiero Customer Lifetime Value

Posted on October 16, 2020December 30, 2020Categories BlogTags , , ,

La industria de los servicios financieros ha cambiado para siempre. La recuperación de la crisis financiera no ha sido fácil; los requisitos de capital son más altos, la supervisión regulatoria es más estricta y los clientes son mas inteligentes.

Para reconstruir la confianza del cliente y garantizar una rentabilidad continua con un riesgo de cartera equilibrado, los analistas de servicios financieros necesitan una visión operativa mas profunda y una comprensión casi perfecta de sus clientes y prospectos.

Para lograr esta transformación del sector, el uso estratégico de datos y de tecnologías avanzadas de Aprendizaje Automático (Machine Learning) son fundamentales, razón por la cual estaremos presentando una visión del uso estratégicos de datos en el sector financiero en nuestro seminario denominado “Cálculo del valor de un cliente” (Customer Lifetime Value) utilizando Aprendizaje Automático con Alteryx.

Demistifying Price Elasticity

Posted on September 20, 2020December 30, 2020Categories BlogTags , , , 1 Comment on Demistifying Price Elasticity

If you missed it, you can see the whole workshop here. (NOTE: The video is in Spanish).

Establecer un precio correcto de productos / servicios es una de las decisiones más importantes que una empresa puede/debe tomar. La subvaloración y/o la sobrevaloración de ellos, pueden perjudicar drásticamente los resultados de una empresa. Entonces, ¿dónde está el punto óptimo, el precio correcto, que maximiza los ingresos y las ganancias? En esta sesión, veremos cómo dar respuesta a estas preguntas, a través de la construcción de modelos para predecir la demanda, la variabilidad estacional y al finalizar la sesión tendremos una visión más completa del problema, su solución teórica y cómo a través del uso de tecnología simple para el usuario podemos resolver prácticamente nuestro caso.

Temas a cubrir:

– Predicción de la demanda.

– Ajustes por estacionalidad.

– Determinación del precio más conveniente para cada período.

Cloud Data Platforms for dummies

Posted on June 7, 2020June 7, 2020Categories Blog

GET THE MOST OUT OF YOUR DATA WITH A CLOUD DATA PLATFORM

Learn how to unite all your data, workloads, insights, and users with a single data platform.

This book reveals how organizations of any size can easily and securely store, integrate, analyze, share, and even monetize their data across multiple clouds and regions for a business without data barriers. By downloading this book you’ll also get insight into: What it means to have a single platform for all your data workloads The benefits of a multi-cluster shared data architecture The problem with traditional data management approaches Securing and governing your data How to calculate hard costs and assessing time to value

10 Essential criteria in evaluating an Analytic Process Automation platform

Posted on June 7, 2020June 7, 2020Categories Blog

THE PROVEN ACCELERANT FOR DIGITAL TRANSFORMATION

The need to amplify human output and upskill people, automate complex and tedious business processes, and democratize access to data and analytics in one unified approach has driven the emergence of a new category of software called Analytic Process Automation (APA). APA platforms have been proven across organizations globally to accelerate data-driven business outcomes across every line of business and industry vertical. But not all Analytic Process Automation platforms are alike.

In order to achieve quick wins and transformative outcomes across the organization, look for these ten criteria when evaluating platforms.

Download eBook…

Data Innovators in Government

Posted on June 7, 2020September 4, 2020Categories Blog

Explore modern approaches for unleashing data-driven
decision-making for Federal, State, and Local governments.


In government–whether local or federal–the pressure to provide faster insights on your agency’s programs rests squarely on your team’s shoulders. Although we live and work in a world with an overabundance of data, quick access to information and insights can be impossible if you struggle with a lack of resources, budget, and legacy systems.

Download Ebook…

 

Alteryx Promote: Data Science Model Production System

Posted on June 5, 2020June 5, 2020Categories BlogTags , , ,   Leave a comment on Alteryx Promote: Data Science Model Production System

Alteryx Promote provides end-to-end data science system for developing, deploying and managing predictive models and real-time decision APIs. It allows data scientists and analytics teams to build, manage and deploy predictive models to production faster — and more reliably — without writing any custom deployment code.

Building predictive models is hard work. Deploying them shouldn’t be.

The typical cycle of deploying predictive models can be an error prone, labor intensive, multi-step approach that can take weeks or even months to deploy- and at the same time costing organizations tens to hundreds of thousands of dollars.

Deploy Deploy Predictive models into production Alteryx Promote makes deploying a predictive model easy with the click of a button regardless of your skillset. Alteryx Promote – Flexibility for Data Scientists and Business Analysts Flexibility for Data Scientists and Business Analysts Delivering R and Python models is immediately accessible via standard REST API without recoding or utilize the code free environment of Alteryx Designer to build and deploy models.

Embed Models with Ease
Embedding models into production apps is as simple as sending an auto-generated code snippet to your dev team. Embed predictive models in any application capable of making REST API requests.
Real-time or Batch
Execute real-time predictions in low-latency applications or use batch mode for bulk offline scoring.