I can confirm this bug.
Edit:

I believe I've figured out the issue to reproduce do the following

1. Enter any area that has a 60 second count down timer (for example I triggered this when I went to fight Edric)
2. No matter what you choose at this point the ability will be broken
3. Note: If you hit decline it appears that the ability to travel back to town no longer works
4. Note: If you hit decline the ability seems to break so I do not believe the issue is with the warp but is instead with the warp message itself.
   ​
   ​
   Original post:
   I don't have reproduction steps yet but I'm trying to figure them out. Here's what I know so far
5. I tend to notice it after I fast travel back FROM town (after having used town portal)
6. It doesn't occur if you login; go somewhere outside of town, use town portal ability, travel to town, return to the location you were at (or any other location via fast travel)
7. It DOES occur after I've played for a while

I think it might be a two stage bug... Something like View a cutscene and then travel back to town. Or possibly, finish a quest and travel to town.

The only way to resolve the issue once it occurs is to exit the game and relaunch it. Returning to main menu -doesn't- resolve it.