The article consists of two connected, but distinct parts. The first concerns certain problems arising in logical paraphrase, which are expressed in natural language. The second part presents some of the logical aspects of the dialogue between theism and atheism. This problem was actually the inspiration for the article itself. In certain papers, discussions, and even in books of logic, I found some interesting, yet dangerous mistakes when applying logic to the analysis of natural or philosophical language. In this article, I propose three differentiated meanings of paraphrase using material implication. Further, I discuss certain implications arising from the distinction, especially those of negation. Analogical survey is next applied in relation to other logical connections: disjunction, exclusive and inclusive disjunction. Neglecting these three different meanings of logical paraphrase can lead to serious misconceptions. A few examples are presented in both parts of the article. In the second part, I construct a logical theory involved in the discourse between theism and atheism. I also include some resulting examples of misplaced logical paraphrase.