Of course our grand concern should not be to take Scripture literally or non-literally. Insofar as we claim to be Christians, our goal must be to take Scripture as Christ took it. Otherwise, we might as well claim to be "SpongeBobians" as "Christians."
Jesus lodged a great many charges against the religious leaders of His day. He complained that their righteousness fell short of that of the Kingdom (Matthew 5:20), that they didn't practice what they preached (Matthew 23:3), that they made proselytes who were worse than they (Matthew 23:15), and a host of other accusations.
But did Jesus ever fault them for being too literal?
I'd say the opposite is the case. If anything, Jesus faulted them for not attending closely enough to the details Scripture.
For instance, He faulted them for failing to learn from David's history (Matthew 12:2-4) and from the practice of the priests (v. 5). He did not suggest that either was a myth or a cultic legend with a general meaning that bypassed the text. It was from the text that He derived the meaning.
Again and again Jesus clashed with the Pharisees' traditionalism. But His problem with this plague was not that it harped too closely on the letter of the Torah, but that it completely ignored it at will, as He illustrated at length in Matthew 15:1-9. Jesus in no way suggested that the Pharisees needed to loosen their grip on Scripture; rather, they needed to tighten it (vv. 3-4, 6, 9).
Jesus accuses the Sadducees of not having a sufficient grasp of Scripture: “You are wrong, because you know neither the Scriptures nor the power of God" (Matthew 22:29). The He presses a seemingly minor point of syntax (v. 32) to demonstrate the resurrection (v. 31).
Then He rounds on the Pharisees, pressing a literal reading of the title of Psalm 110 to make a major point which would collapse otherwise (22:41-45). Jesus took "of David" to mean "of David," and not "David-like" or "from the Davidic school."
“Have you not read that he who created them from the beginning made them male and female [ Genesis 1:27], and said, ‘Therefore a man shall leave his father and his mother and hold fast to his wife, and the two shall become one flesh’ [Genesis 2:24]? So they are no longer two but one flesh. What therefore God has joined together, let not man separate.” (Matthew 19:4-5)
You search the Scriptures because you think that in them you have eternal life; and it is they that bear witness about me, yet you refuse to come to me that you may have life. ...Do not think that I will accuse you to the Father. There is one who accuses you: Moses, on whom you have set your hope. For if you believed Moses, you would believe me; for he wrote of me. But if you do not believe his writings, how will you believe my words?" (John 5:39-40, 45-47)