I'm implementing an application which records and analyzes audio in real time (or at least as close to real time as possible), using the JDK Version 8 Update 201. While performing a test which simulates typical use cases of the application, I noticed that after several hours of recording audio continuously, a sudden delay of somewhere between one and two seconds was introduced. Up until this point there was no noticeable delay. It was only after this critical point of recording for several hours when this delay started to occur.
What I've tried so far
To check if my code for timing the recording of the audio samples is wrong, I commented out everything related to timing. This left me essentially with this update loop which fetches audio samples as soon as they are ready (Note: Kotlin code):
while (!isInterrupted) {
val audioData = read(sampleSize, false)
listener.audioFrameCaptured(audioData)
}
这是我的读取方法:
fun read(samples: Int, buffered: Boolean = true): AudioData {
//Allocate a byte array in which the read audio samples will be stored.
val bytesToRead = samples * format.frameSize
val data = ByteArray(bytesToRead)
//Calculate the maximum amount of bytes to read during each iteration.
val bufferSize = (line.bufferSize / BUFFER_SIZE_DIVIDEND / format.frameSize).roundToInt() * format.frameSize
val maxBytesPerCycle = if (buffered) bufferSize else bytesToRead
//Read the audio data in one or multiple iterations.
var bytesRead = 0
while (bytesRead < bytesToRead) {
bytesRead += (line as TargetDataLine).read(data, bytesRead, min(maxBytesPerCycle, bytesToRead - bytesRead))
}
return AudioData(data, format)
}
然而,即使没有我这边的任何时间,问题也没有得到解决.因此,我继续进行了一些实验,让应用程序使用不同的音频格式运行,这会导致非常混乱的结果(除非另有规定,否则我将使用带有little endian的PCM签名16位立体声音频格式,默认采样率为44100.0 Hz):
- 延迟出现之前必须经过的临界时间似乎因使用的机器不同而有所不同.在我的Windows10台式PC上,这个时间在6.5到7个小时之间.然而,在我的笔记本电脑(也使用Windows10)上,同样的音频格式需要4到5个小时.
- The amount of audio channels used seems to have an effect. If I change the amount of channels from stereo to mono, the time before the delay starts to appear is doubled to somewhere between 13 and 13.5 hours on my desktop.
- 将采样大小从16位减少到8位还会导致延迟开始出现之前的时间加倍.在我的桌面上呆了13到13.5个小时.
- 将字节顺序从小端更改为大端没有任何影响.
- Switching from stereomix to a physical microphone has no effect either.
- I tried opening the line using different buffer sizes (1024, 2048 and 3072 sample frames) as well as its default buffer size. This also didn't change anything.
- 刷新TargetDataLine after,延迟已经开始发生,导致所有字节在大约一到两秒内为零.在此之后,我再次得到非零值.然而,延迟仍然存在.如果我刷新临界点before行,我得不到那些零字节.
- 停止并重新启动TargetDataLine after出现的延迟也不会改变任何事情.
- Closing and reopening the TargetDataLine, however, does get rid of the delay until it reappears after several hours from there on.
- Automatically flushing the TargetDataLines internal buffer every ten minutes does not help to resolve the issue. Therefore, a buffer overflow in the internal buffer does not seem to be the cause.
- 使用并行垃圾收集器来避免应用程序冻结也无济于事.
- 使用的采样率似乎很重要.如果我将采样率加倍到88200赫兹,延迟开始出现在运行时间的3到3.5小时之间.
- If I let it run under Linux using my "default" audio format, it still runs fine after about 9 hours of runtime.
Conclusions that I've drawn:
These results let me come to the conclusion that the time for which I can record audio before this issue starts to happen is dependent on the machine on which the application is run and dependent on the byte rate (i.e. frame size and sample rate) of the audio format. This seems to hold true (although I can't completely confirm this as of now) because if I combine the changes made in 2 and 3, I would assume that I can record audio samples for four times as long (which would be somewhere between 26 and 27 hours) as when using my "default" audio format before the delay starts to appear. As I didn't find the time to let the application run for this long yet, I can only tell that it did run fine for about 15 hours before I had to stop it due to time constraints on my side. So, this hypothesis is still to be confirmed or denied.
根据项目符号13的结果,似乎整个问题只有在使用Windows时才会出现.因此,我think认为它might是javax.sound.sampled API的平台特定部分中的错误.
Even though I think I might have found a way to change when this issue starts to happen, I'm not satisfied with the result. I could periodically close and reopen the line to avoid the problem from starting to appear at all. However, doing this would result in some arbitrary small amount of time where I wouldn't be able to capture audio samples. Furthermore, the Javadoc states that some lines can't be reopened at all after being closed. Therefore, this is not a good solution in my case.
Ideally, this whole issue shouldn't be happening at all. Is there something I am completely missing or am I experiencing limitations of what is possible with the javax.sound.sampled API? How can I get rid of this issue at all?
Edit: By suggestion of Xtreme Biker and gidds I created a small example application. You can find it inside this Github repository.