Fig tree leaves turning brown and shriveling

Fig tree leaves turning rusty brown, shrivelling and decaying. First noticed when I took it outside in May so brought it in and put it back out recently. It does well inside by a window but I want it to…