I make the call based on whether I’m interested in maintaining a relationship with that person outside of work. The answer is usually “No.” Outside-the-job relationships tend to complicate things at times and can raise expectations more than actually warranted. If things go wrong at work, then you have the added issue of how that impacts the friendship, real or digital. Generally speaking, it is not a good idea in my opinion.
The good news is that it typically requires a positive response on your part for the connection to happen. A simple failure to confirm/accept will keep the link from being made. If you realize that you’ve made a mistake, you can undo things (un-friend) without awkwardness as the rejected party receives no notification.