Paper review:
Adaptive Playout Mechanisms for Packetized Audio Application in Wide-Area
Networks
Reviewer:
Mike Liu
- State the problem the paper is trying to solve.
The main problem the paper is trying to solve is how to develop and
evaluate different algorithms for adaptively adjusting the playout delay
of audio packets in an interactive packet-audio terminal application, in
the face of varying network delays.
- State the main contribution of the paper: solving a new problem, proposing a
new algorithm, or presenting a new evaluation (analysis). If a new problem, why
was the problem important? Is the problem still important today? Will the
problem be important tomorrow? If a new algorithm or new
evaluation (analysis), what are the improvements over previous algorithms or
evaluations? How do they come up with the new algorithm or evaluation?
The main contribution of this paper is that the authors investigated and
evaluated the performance of four different algorithms for adaptively adjusting
the playout delay for audio packets in the face of varying network delays.
This is a new analysis because adaptively adjusting to delay distribution by
adjusting the playout delay is relatively new technique and much work had not
been down to evaluate the performance of different algorithms of this type. This
evaluation is an improvement over just presenting a single algorithm that has
this new feature because by comparing several algorithms of this type, more
knowledge is gained regarding which algorithm should be implemented. The problem
will continue to be important as the Internet welcomes more and more real-time,
interactive multimedia applications.
- Summarize the (at most) 3 key main ideas (each in 1 sentence.)
The three 3 key main ideas are:
(1) An approach to dealing with the unknown nature of the delay distribution is
to estimate these delays and adaptively respond to their change by dynamically
adjusting the playout delay.
(2) Four algorithms of this type were tested and evaluated using the
experimentally-obtained delay measurements of audio traffic between several
different Internet sites.
(3) In comparing the algorithms, the results indicated that an adaptive algorithm
which explicitly adjust to the sharp, spike-like increases in packet delay which
were observed in traces can achieve a lower rate of lost packets for both a given
average playout delay and a given maximum buffer size.
- Critique the main contribution
- Rate the significance of the paper on a scale of 5
(breakthrough), 4 (significant contribution), 3 (modest contribution), 2
(incremental contribution), 1 (no contribution or negative contribution).
Explain your rating in a sentence or two.
I give this paper a rating of 4 because it provides needed evaluation of
algorithms that may prove to be quite popular and widespread in audio
applications on the Internet in the near future. The paper also provided its
own algorithm for comparison against other similar algorithms, in addition to
the process of evaluating such algorithms.
- Rate how convincing the methodology is: how do the authors justify the solution
approach or evaluation? Do the authors use arguments, analyses, experiments,
simulations, or a combination of them? Do the claims and conclusions follow from the
arguments, analyses or experiments? Are the assumptions realistic (at the time of the
research)? Are the assumptions still valid today? Are the experiments well designed?
Are there different experiments that would be more convincing? Are there other
alternatives the authors should have considered? (And, of course, is the paper free of
smethodological errors.)
Their methodology was pretty convincing. However, it would have been nice if
more results were displayed and some of the drawbacks of using their
Algorithm 4 were described to show the tradeoffs of employing their
algorithm. To compare the performance of the four algorithms, the authors
wrote a simulator which takes the received packet trace file generated by
ruuning NeVot and simulates the behavior of the playout algorithms. It was
through these experiments on simulators using data from the real Internet
that produced the results used for comparison. This was a nice tradeoff
because they could not produce the exact same conditions on the Internet for
all the algorithm tests so they took one snapshot of the Internet and had the
simulator feed each of the algoritms the same Internet conditions so they all
ran under identical network conditions.
- What is the most important limitation of the approach?
The most important limitation of their experimental approach is that
ultimately the test were run a simulator and not on the real Internet and
though this is better in terms of controling for different network
variations, their tests are only as good as their simulator. Also, it would
be useful to know if the sharp, spike-like increases in packet delay which
the authors observed in their traces were inherent in their simulator or in
their connection to the internet, or if these delays are typical of what most
users of their algorithm would experience since the fourth algorithm gets
most of its performance improvements from correcting for this condition.
- What lessons should researchers and builders take away from this work. What
(if any) questions does this work leave open?
The lessons that researchers should take away from this work are that an
adaptive algorithm which explicitly adjusts to the sharp, spike-like
increases in packet delay can achieve a lower rate of lost packets for both a
given average playout delay and a given maximum buffer size. The questions
the work leaves open are what are the timed trial results and average packet
losses on the real Internet like for these four algorithms and are the
results the same. Also, another area for investigation are the possibilities
for quantifying the distortion which occurs when silence periods are
artifically contracted or expanded, and how can the different algorithms be
compared based on these factors. Finally, additional research is needed in
formalizing and arriving at a better understanding of the dynamics of network
delays.