В PHP различие такое:
Variable parsing
When a string is specified in double quotes or with heredoc, variables are parsed within it.
There are two types of syntax, a simple one and a complex one. The simple syntax is the most common and convenient, it provides a way to parse a variable, an array value, or an object property.
The complex syntax was introduced in PHP 4, and can be recognised by the curly braces surrounding the expression.
На деле это выглядит так:
$test = 'test';
$s1 = 'this is the $test'; // this is the $test
$s2 = "this is the $test"; // this is the test
Использование " или ' сказывается на скорости. По поводу этого можно почитать статью
"Оптимизация программ на PHP".