The Thylacine looked closer to a dog, and was around 8-30kg. They were around 100-130cm long, not including the tail. It went extinct relatively recently
The Thylacoleo looked more like a Lioness, and was around 100-130kg, and 150cm long - again not including the tail. They also had clawed thumbs, and bizarre looking, yet highly effective teeth that likely gave it the strongest bite of any mammal. They went extinct around the time the first Aboriginals arrived, due to competition for food, unlike the Thylacine, which was able to survive until recently on Tasmania.
The Thylacine's closest relatives are the Quolls and other Dasyuromorphians. The Thylacoleo's are Wombats and Koalas (They are thought to be the inspiration behind Drop Bears). Their last common ancestor likely lived around 25-50 Million years ago, I couldn't find a more precise range