Latency-sensitive applications, such as augmented reality, industrial automation, distributed gaming, real-time analytics and intelligent dialog systems are on the rise, largely promoted by the high-speed connectivity of upcoming 5G networks. One commonality of these applications is that they usually require a video or speech streaming analytics microservice. A second commonality is the requirement of a low execution latency and low latency variance. This talk discusses the design option and challenges when including hardware accelerators in edge cloud infrastructures in order to reach these tight latency constraints.