If you forget the answer of the security question, the only method to go back to OS is to access Recovery mode and do hardware reset on your phone.
How to perform hardware reset: http://www.asus.com/support/FAQ/1006346 Note: All data stored on your phone will be deleted after reset.