Implementing the Whisper API for SPR: A Journey Through Speech-to-Text and Beyond
Hello fellow developers and tech enthusiasts! 🌟
Today, I want to share my experience implementing the Whisper API for SPR. But first, let's answer the question that might be on your mind:
What is Whisper? 🤔
Whisper is, arguably, the world's most advanced "Speech-to-Text" system. Here's why it's so cool:
* 🗨� No Training Required: Just speak, and it writes down what you say.
* 📝 GPT-3 Integration: You can even have GPT-3 correct the transcribed text.
* 🌍 Multilingual Support: It understands nearly 100 languages and can translate them to English or transcribe as is.
Local vs. Online Versions 🏠☁️
* Local Version: Requires installation and is resource-intensive.
* Online Version: A cost-effective cloud service by OpenAI. This is the version we're using here.
Challenges and Solutions 🛠�
Implementing Whisper wasn't a walk in the park, especially with Powerbasic's limitations, such as:
* Web Access and Proxy Servers
* Lack of support for Multi-Part HTTP Requests
I initially found some code on the Purebasic Forum, but it wasn't plug-and-play. After diving deep into Multi-Part Requests, I managed to get it working.
Customization and Versatility 🎛�
I made several changes to adapt the library for my specific task: implementing OpenAI's Whisper API. Now that it's compatible with OpenAI, you can use this library for various other API-related tasks.
Note: I used some global variables, so it's not ready to run "out of the box."
How to Use Whisper with an .mp3 File 🎵
To call Whisper with an ".mp3 file," you would proceed as follows:
; Initialize a new multipart form data request
m = MPFR_Create()
; Add the MP3 data to the request
; MPFR_AddDataFile(m,"file",*mp3Memory, mp3Size,"Zahlen_PCM16.wav", MimeType("audio/wav"))
; S03 - Filename mit Endung
S04=MimeType(S03)
MPFR_AddDataFile(m,"file",*mp3Memory, mp3Size,S03, S04)
MPFR_AddTextField(m, "model", modelID)
; Send the request to the OpenAI API in headers
status = MPFR_Send_with_API_Key(m, endP)
serverResponse = MPFRequest(m)\response_message
If status = #False
AI_Return=serverResponse
Set_Error("Error: Failed to send HTTP request")
Goto ExitPoint
EndIf
Below is the Library for Purebasic.
That's it for now! If you have any questions or need further clarification, feel free to ask.
Happy coding! 🚀
; https://www.purebasic.fr/english/viewtopic.php?f=12&t=77006
;
; XIncludeFile "MultipartFormDataRequests.pbi"
;
; img = CreateImage(#PB_Any,400,400,32,#Blue)
;
; UseJPEGImageEncoder()
; *ImgBuffer = EncodeImage(img,#PB_ImagePlugin_JPEG)
; ImageLen.i = MemorySize(*ImgBuffer)
;
; m.i = MPFR_Create()
; MPFR_AddData(m,"filedata",*ImgBuffer,ImageLen,MimeTypeForFile("1.jpg"))
; MPFR_AddTextField(m,"onlinefn","my_online_file.jpg")
; If MPFR_Send(m,"https://www.mywebsite.com/MPF_File_Receiver.php")
; wb.i = Val(StringField(MPFRequest(m)\response_message,2,c32))
; If wb = ImageLen
; Debug "success"
; EndIf
; EndIf
; MPFR_Free(m)
;
; FreeMemory(*ImgBuffer)
;--------------------------------------------------------------------
;
;--------------------------------------------------------------------
Global c34.s{1} = Chr(34)
;--------------------------------------------------------------------
;
;--------------------------------------------------------------------
CompilerIf Defined(RandomCharacters,#PB_Procedure)=0
;--------------------------------------------------------------------
;
;--------------------------------------------------------------------
Macro RandomLetter()
Chr(65+Random(25))
EndMacro
;--------------------------------------------------------------------
;
;--------------------------------------------------------------------
Procedure.s RandomCharacters(l.i)
t.s
If l>0
For a = 1 To l
Select Random(2)
Case 0
t + Str(Random(9))
Case 1
t + RandomLetter()
Case 2
t + LCase(RandomLetter())
EndSelect
Next a
EndIf
ProcedureReturn t
EndProcedure
;--------------------------------------------------------------------
;
;--------------------------------------------------------------------
Procedure.s FormatURLParameter(t.s)
t = ReplaceString(t,"?","%3F")
t = ReplaceString(t,"/","%2F")
t = URLEncoder(t)
ProcedureReturn t
EndProcedure
;--------------------------------------------------------------------
;
;--------------------------------------------------------------------
CompilerEndIf
;--------------------------------------------------------------------
;
;--------------------------------------------------------------------
Procedure.s MimeType(ext.s)
ext = RemoveString(LCase(ext),".")
Select ext
Case "txt"
ProcedureReturn "text/plain"
Case "json"
ProcedureReturn "application/json"
Case "xml"
ProcedureReturn "application/xml"
Case "mp3", "ogg", "m4a","wav","oga","flac","mpga"
ProcedureReturn "audio/"+ext
Case "jpg", "jpeg"
ProcedureReturn "image/jpeg"
Case "png", "gif", "webp"
ProcedureReturn "image/"+ext
Case "mp4", "webm","mpeg"
ProcedureReturn "video/"+ext
EndSelect
EndProcedure
;--------------------------------------------------------------------
;
;--------------------------------------------------------------------
Enumeration 1
#MPFR_FieldType_File
#MPFR_FieldType_Data
#MPFR_FieldType_Text
#MPFR_FieldType_Data_File
EndEnumeration
;--------------------------------------------------------------------
;
;--------------------------------------------------------------------
Structure MPFR_Field_Structure
field_type.b
Map field_parameter.s()
Post.s
PostLen.i
ContentLen.i
filename.s
field_name.s
text.s
fn.s
*buffer
DataLen.i
mime.s
EndStructure
;--------------------------------------------------------------------
;
;--------------------------------------------------------------------
Structure MPFR_URLParameter_Structure
name.s
value.s
EndStructure
;--------------------------------------------------------------------
;
;--------------------------------------------------------------------
Structure MPFRequestStructure
active.b
List url_parameter.MPFR_URLParameter_Structure()
List field.MPFR_Field_Structure()
response_message.s
status_code.i
error_message.s
EndStructure
;--------------------------------------------------------------------
;
;--------------------------------------------------------------------
Global Dim MPFRequest.MPFRequestStructure(1)
;--------------------------------------------------------------------
;
;--------------------------------------------------------------------
Procedure.i MPFR_Create()
reqs = ArraySize(MPFRequest())
For q = 1 To reqs
If Not MPFRequest(q)\active
MPFRequest(q)\active = #True
ProcedureReturn q
EndIf
Next q
reqs+1 : ReDim MPFRequest(reqs)
MPFRequest(reqs)\active = #True
ProcedureReturn reqs
EndProcedure
;--------------------------------------------------------------------
;
;--------------------------------------------------------------------
Procedure.b MPFR_AddFile(q.i,field_name.s,fn.s)
AddElement(MPFRequest(q)\field())
With MPFRequest(q)\field()
\field_type = #MPFR_FieldType_File
\field_parameter("name") = field_name
\fn = fn
;\mime = MimeTypeForFile(fn)
EndWith
EndProcedure
;--------------------------------------------------------------------
;
;--------------------------------------------------------------------
Procedure.b MPFR_AddTextField(q.i,field_name.s,txt.s)
AddElement(MPFRequest(q)\field())
With MPFRequest(q)\field()
\field_type = #MPFR_FieldType_Text
\field_parameter("name") = field_name
\text = txt
;R("TEXT: *"+txt+"*")
EndWith
EndProcedure
;--------------------------------------------------------------------
;
;--------------------------------------------------------------------
Procedure.b MPFR_AddData(q.i,field_name.s,*SourceBuffer,datalen.i,mime.s)
AddElement(MPFRequest(q)\field())
With MPFRequest(q)\field()
\field_type = #MPFR_FieldType_Data
\field_parameter("name") = field_name
\buffer = AllocateMemory(datalen+2)
CopyMemory(*SourceBuffer,\buffer,datalen)
\DataLen = datalen
\field_name = field_name
\mime = mime
EndWith
EndProcedure
;--------------------------------------------------------------------
;
;--------------------------------------------------------------------
Procedure.b MPFR_AddDataFile(q.i, field_name.s, *SourceBuffer, dataLen.l, filename.s, mime.s)
AddElement(MPFRequest(q)\field())
With MPFRequest(q)\field()
\field_type = #MPFR_FieldType_Data_File ; New field type
\field_parameter("name") = field_name ; Name of the field
\buffer = AllocateMemory(datalen+2)
CopyMemory(*SourceBuffer,\buffer,datalen) ; Pointer to the binary data
\DataLen = DataLen ; Length of the binary data
\filename = filename ; Name of the file
\field_name = field_name
\mime = mime ; MIME type of the file
EndWith
EndProcedure
;--------------------------------------------------------------------
;
;--------------------------------------------------------------------
Procedure.b MPFR_AddJSON(q.i,field_name.s,js.i)
AddElement(MPFRequest(q)\field())
With MPFRequest(q)\field()
\field_type = #MPFR_FieldType_Data
\mime = MimeType("json")
\field_parameter("name") = field_name
t.s = ComposeJSON(js)
\DataLen = StringByteLength(t,#PB_UTF8)
\buffer = AllocateMemory(\DataLen+2)
PokeS(\buffer,t,-1,#PB_UTF8)
EndWith
EndProcedure
;--------------------------------------------------------------------
;
;--------------------------------------------------------------------
Procedure.b MPFR_AddXML(q.i,field_name.s,xm.i)
AddElement(MPFRequest(q)\field())
With MPFRequest(q)\field()
\field_type = #MPFR_FieldType_Data
\mime = MimeType("xml")
\field_parameter("name") = field_name
\DataLen = ExportXMLSize(xm)
\buffer = AllocateMemory(\DataLen+2)
ExportXML(xm,\buffer,\DataLen)
EndWith
EndProcedure
;--------------------------------------------------------------------
;
;--------------------------------------------------------------------
Procedure.b MPFR_AddFieldParameter(q.i,attrname.s,attrvalue.s)
MPFRequest(q)\field()\field_parameter(attrname) = attrvalue
EndProcedure
;--------------------------------------------------------------------
;
;--------------------------------------------------------------------
Procedure.b MPFR_AddURLParameter(q.i,param_name.s,param_value.s)
AddElement(MPFRequest(q)\url_parameter())
With MPFRequest(q)\url_parameter()
\name = param_name
\value = param_value
EndWith
EndProcedure
;--------------------------------------------------------------------
;
;--------------------------------------------------------------------
Procedure.b MPFR_Send(q.i,raw_u.s)
With MPFRequest(q)
Repeat
boundary.s = RandomCharacters(50)
BoundaryLen.i = StringByteLength(boundary, #PB_UTF8)
;R(boundary+c13+"LEN: "+Str(BoundaryLen))
If BoundaryLen>70 : Continue : EndIf
violation.b = #False
ForEach \field()
If \field()\field_type=#MPFR_FieldType_Text And FindString(\field()\text,boundary,0,#PB_String_NoCase)
violation = #True
Break
EndIf
Next
Until Not violation
TotalBufferSize.i = 0
ForEach \field()
\field()\Post = #CRLF$ + "--" + boundary + #CRLF$
;\field()\Post + "Content-Disposition: form-data; name="+c34+\field()\field_name+c34 + #CRLF$
;\field()\Post + "Content-Disposition: form-data; name="+c34+\field()\field_name+c34+"; filename="+c34+"give_filename.jpg"+c34+"; " + #CRLF$
cdfldarr.s = ""
ForEach \field()\field_parameter()
cdfldarr + MapKey(\field()\field_parameter())+"="+c34+\field()\field_parameter()+c34+"; "
Next
\field()\Post + "Content-Disposition: form-data; " + cdfldarr + #CRLF$
Select \field()\field_type
Case #MPFR_FieldType_File
\field()\Post + "Content-Type: "+MimeType(GetExtensionPart(\field()\fn)) + #CRLF$
\field()\ContentLen = FileSize(\field()\fn)
Case #MPFR_FieldType_Data
\field()\Post + "Content-Type: "+\field()\mime + #CRLF$
\field()\ContentLen = \field()\DataLen
Case #MPFR_FieldType_Text
\field()\Post + "Content-Type: "+MimeType("txt") + #CRLF$
\field()\ContentLen = StringByteLength(\field()\text,#PB_UTF8)
Case #MPFR_FieldType_Data_File
\field()\Post + "Content-Disposition: form-data; name="+c34+\field()\field_name+c34+"; filename="+c34+"give_filename.jpg"+c34+"; " + #CRLF$
\field()\Post + "Content-Type: image/jpeg" + #CRLF$ ; You can modify this MIME type based on the actual file type.
\field()\ContentLen = \field()\DataLen ; Assuming DataLen contains the length of the binary data
Default
MessageRequester("Error","Don't know what to do with field type "+Str(\field()\field_type))
EndSelect
\field()\Post + #CRLF$
\field()\PostLen = StringByteLength(\field()\Post, #PB_UTF8)
TotalBufferSize + \field()\PostLen + \field()\ContentLen
Next
TotalBufferSize + 2+2+BoundaryLen+2+2
;R("TOTAL SIZE FOR BUFFER: "+StrD( TotalBufferSize /1000/1000) )
*Buffer = AllocateMemory(TotalBufferSize, #PB_Memory_NoClear)
If Not *Buffer
Debug "failure to allocate memory"
ProcedureReturn #False
EndIf
*BufferPosition = *Buffer
ForEach \field()
PokeS(*BufferPosition, \field()\Post, -1, #PB_UTF8|#PB_String_NoZero)
*BufferPosition + \field()\PostLen
Select \field()\field_type
Case #MPFR_FieldType_File
fs = FileSize(\field()\fn)
f = ReadFile(#PB_Any,\field()\fn,#PB_File_SharedRead)
;R(\field()\fn+c13+"SIZE: "+Str(fs)+c13+"F: "+Str(f))
ReadData(f,*BufferPosition,\field()\ContentLen)
CloseFile(f)
Case #MPFR_FieldType_Data
CopyMemory(\field()\buffer, *BufferPosition, \field()\ContentLen)
Case #MPFR_FieldType_Text
PokeS(*BufferPosition, \field()\text, -1, #PB_UTF8|#PB_String_NoZero)
Case #MPFR_FieldType_Data_File
CopyMemory(\field()\buffer, *BufferPosition, \field()\ContentLen)
EndSelect
*BufferPosition + \field()\ContentLen
Next
PokeS(*BufferPosition, #CRLF$ + "--" + boundary + "--" + #CRLF$, -1, #PB_UTF8|#PB_String_NoZero)
*BufferPosition + (2+2+BoundaryLen+2+2)
u.s = raw_u
If ListSize(\url_parameter())
u + "?"
FirstElement(\url_parameter())
u + \url_parameter()\name+"=" + FormatURLParameter(\url_parameter()\value)
While NextElement(\url_parameter())
u + "&"+\url_parameter()\name+"=" + FormatURLParameter(\url_parameter()\value)
Wend
EndIf
NewMap Header.s()
Header("Content-Type") = "multipart/form-data; boundary=" + boundary
Header("Content-Length") = Str(TotalBufferSize)
HttpRequest.i = HTTPRequestMemory(#PB_HTTP_Post, u, *Buffer, TotalBufferSize, 0, Header())
If HttpRequest
\response_message = HTTPInfo(HTTPRequest, #PB_HTTP_Response)
\status_code = Val(HTTPInfo(HTTPRequest,#PB_HTTP_StatusCode))
\error_message = HTTPInfo(HTTPRequest,#PB_HTTP_ErrorMessage)
;MBX("Error:"+\error_message)
;MBX("Response:"+\response_message)
If \status_code=200
;MBX( "RT: *"+\response_message+"*")
status.b = #True
Else
Set_Error("Response: *"+\response_message+"*")
Set_Error("ERROR: "+\error_message+"*")
EndIf
FinishHTTP(HttpRequest)
EndIf
FreeMemory(*Buffer)
ProcedureReturn status
EndWith
EndProcedure
;--------------------------------------------------------------------
;
;--------------------------------------------------------------------
; U01 -\field()\fn
; U01 - Dateiname
Procedure.s MPFR_GetContent_Type(U01.s="")
Protected.s S02,S03
S02=Trim(U01)
If Len(U01)=0
S02=MPFRequest(q)\field()\fn
EndIf
If FindString(S02,"\")>0
S02=GetFilePart(S02)
EndIf
S03="Content-Type: "+MimeType(GetExtensionPart(S02))
ProcedureReturn S03
EndProcedure
;--------------------------------------------------------------------
;
;--------------------------------------------------------------------
; Use the Open AI-API-Key
Procedure.b MPFR_Send_with_API_Key(q.i, raw_u.s)
With MPFRequest(q)
Repeat
boundary.s = RandomCharacters(50)
BoundaryLen.i = StringByteLength(boundary, #PB_UTF8)
;R(boundary+c13+"LEN: "+Str(BoundaryLen))
If BoundaryLen>70 : Continue : EndIf
violation.b = #False
ForEach \field()
If \field()\field_type=#MPFR_FieldType_Text And FindString(\field()\text,boundary,0,#PB_String_NoCase)
violation = #True
Break
EndIf
Next
Until Not violation
TotalBufferSize.i = 0
ForEach \field()
\field()\Post = #CRLF$ + "--" + boundary + #CRLF$
;\field()\Post + "Content-Disposition: form-data; name="+c34+\field()\field_name+c34 + #CRLF$
;\field()\Post + "Content-Disposition: form-data; name="+c34+\field()\field_name+c34+"; filename="+c34+"give_filename.jpg"+c34+"; " + #CRLF$
; Kommentar entfernt
cdfldarr.s = ""
ForEach \field()\field_parameter()
cdfldarr + MapKey(\field()\field_parameter())+"="+c34+\field()\field_parameter()+c34+"; "
Next
\field()\Post + "Content-Disposition: form-data; " + cdfldarr + #CRLF$
Select \field()\field_type
Case #MPFR_FieldType_File
\field()\Post + MPFR_GetContent_Type() + #CRLF$
\field()\ContentLen = FileSize(\field()\fn)
Case #MPFR_FieldType_Data
\field()\Post + "Content-Type: "+\field()\mime + #CRLF$
\field()\ContentLen = \field()\DataLen
Case #MPFR_FieldType_Text
\field()\Post + "Content-Type: "+MimeType("txt") + #CRLF$
\field()\ContentLen = StringByteLength(\field()\text,#PB_UTF8)
Case #MPFR_FieldType_Data_File
\field()\Post = #CRLF$ + "--" + boundary + #CRLF$
\field()\Post + "Content-Disposition: form-data; " + cdfldarr
S02.s= \field()\Post
S02 + "filename="+c34+\field()\filename+c34+"; " + #CRLF$
\field()\Post=S02
S02=MPFR_GetContent_Type(\field()\filename)+#CRLF$ ; You can modify this MIME type based on the actual file type.
\field()\Post + S02
\field()\ContentLen = \field()\DataLen ; Assuming DataLen contains the length of the binary data
;MBX(\field()\Post)
Default
MessageRequester("Error","Don't know what to do with field type "+Str(\field()\field_type))
EndSelect
\field()\Post + #CRLF$
\field()\PostLen = StringByteLength(\field()\Post, #PB_UTF8)
TotalBufferSize + \field()\PostLen + \field()\ContentLen
Next
TotalBufferSize + 2+2+BoundaryLen+2+2
;R("TOTAL SIZE FOR BUFFER: "+StrD( TotalBufferSize /1000/1000) )
*Buffer = AllocateMemory(TotalBufferSize, #PB_Memory_NoClear)
If Not *Buffer
Debug "failure to allocate memory"
ProcedureReturn #False
EndIf
*BufferPosition = *Buffer
ForEach \field()
PokeS(*BufferPosition, \field()\Post, -1, #PB_UTF8|#PB_String_NoZero)
*BufferPosition + \field()\PostLen
Select \field()\field_type
Case #MPFR_FieldType_File
fs = FileSize(\field()\fn)
f = ReadFile(#PB_Any,\field()\fn,#PB_File_SharedRead)
;R(\field()\fn+c13+"SIZE: "+Str(fs)+c13+"F: "+Str(f))
ReadData(f,*BufferPosition,\field()\ContentLen)
CloseFile(f)
Case #MPFR_FieldType_Data
CopyMemory(\field()\buffer, *BufferPosition, \field()\ContentLen)
Case #MPFR_FieldType_Text
PokeS(*BufferPosition, \field()\text, -1, #PB_UTF8|#PB_String_NoZero)
Case #MPFR_FieldType_Data_File
CopyMemory(\field()\buffer, *BufferPosition, \field()\ContentLen)
EndSelect
*BufferPosition + \field()\ContentLen
Next
PokeS(*BufferPosition, #CRLF$ + "--" + boundary + "--" + #CRLF$, -1, #PB_UTF8|#PB_String_NoZero)
*BufferPosition + (2+2+BoundaryLen+2+2)
u.s = raw_u
If ListSize(\url_parameter())
u + "?"
FirstElement(\url_parameter())
u + \url_parameter()\name+"=" + FormatURLParameter(\url_parameter()\value)
While NextElement(\url_parameter())
u + "&"+\url_parameter()\name+"=" + FormatURLParameter(\url_parameter()\value)
Wend
EndIf
; Before sending the HTTP request, add the API Key to the headers.
NewMap Header.s()
Header("Content-Type") = "multipart/form-data; boundary=" + boundary
Header("Content-Length") = Str(TotalBufferSize)
Header("Authorization") = "Bearer " + AI_OKEY ; <- Include API Key here
HttpRequest.i = HTTPRequestMemory(#PB_HTTP_Post, u, *Buffer, TotalBufferSize, 0, Header())
;FreeMemory(*Buffer)
If HttpRequest
\response_message = HTTPInfo(HTTPRequest, #PB_HTTP_Response)
\status_code = Val(HTTPInfo(HTTPRequest,#PB_HTTP_StatusCode))
\error_message = HTTPInfo(HTTPRequest,#PB_HTTP_ErrorMessage)
;MBX("Error:"+\error_message)
;SetClipboardText("Error:"+\error_message)
;MBX("Response:"+\response_message)
If \status_code=200
;MBX( "RT: *"+\response_message+"*")
status.b = #True
Else
Set_Error("Response: *"+\response_message+"*")
Set_Error("ERROR: "+\error_message+"*")
EndIf
FinishHTTP(HttpRequest)
EndIf
FreeMemory(*Buffer)
ProcedureReturn status
EndWith
EndProcedure
;--------------------------------------------------------------------
;
;--------------------------------------------------------------------
Procedure.b MPFR_Free(q.i)
With MPFRequest(q)
ForEach \field()
If \field()\buffer<>0
FreeMemory(\field()\buffer)
EndIf
Next
ClearList(\field())
\active = #False
EndWith
EndProcedure
;--------------------------------------------------------------------
;
;--------------------------------------------------------------------