Solved thread

This post is marked as solved. If you think the information contained on this thread must be part of the official documentation, please contribute submitting a pull request to its repository.

Newlines/carriage returns (\r and \n) in textareas

Line breaks/new lines entered in a textarea are stored in MySQL, but when you go back and edit it, the line breaks are gone and all text is on the same line.

The reason is that all new lines and line breaks are filtered out from the output, in public/index.php:

echo str_replace(["\n","\r","\t"], '', $application->handle()->getContent());

When I remove the str_replace part, line breaks in the textarea are shown as expected.

My question is: why is that str_replace there? Why does the output need to be on one line? I think line breaks can never be shown in a textarea as long as that str_replace is there, right? Or am I missing something?

Well that's only when you generate scaffolding with DevTools?

No, it's not in the code generated by scaffolding. The problem is in the core index.php (front controller) that handles every request. (See Oleg's link to the commit that added str_replace to the project templates.)

All HTML output, from every request, is processed by this str_replace line in index.php. It removes all unnecessary whitespace from the output: newlines, carriage returns and tabs. And that's where the 'bug' is: whitespace must not be removed within <textarea> and <pre> tags, because it is printed literally.

You can easily reproduce the bug by putting this in a view:

<textarea cols=80>
this is line 1
this is line 2
</textarea>

When viewing the output in a browser, you would expect to see two separate lines in the textarea, but instead they will be on one line, which is wrong.

There is no workaround. For instance, <br> is printed literally in a textarea. The only solution is to fix or remove str_replace.

I was working on a regular expression to fix it (works only for <textarea>, not for <pre>):

echo preg_replace(
    "/(\n|\r|\t)(?!(?>[^<]*<(?!textarea\b))*(?>[^<]*)<\/textarea>)/",
    '',
    $application->handle()->getContent()
);

It works, but it is really slow, even though it uses non-backtracking subexpressions for speed. My testpage (~250 kB) takes about 10 seconds to process. :-|

For every \n, \r or \t it will look ahead to check if it is followed by a </textarea> tag and if there's not a <textarea> (opening) tag in between. If HTML tags within a textarea would be formatted like lt/gt instead of </>, I could just look for the next < and check if it's followed by /textarea, that would be fast. But alas, < and > are allowed in textareas, so I have to scan the entire rest of the document for every newline/tab.

But any regex is pointless if Phalcon is slowed down by it. We want it to be superfast! :-)

So the easiest way to fix this is to remove str_replace again. Phalcon's processing will be slightly faster. The increased output size is hardly noticable. For instance, a large 250 kB HTML page may grow 5%, but on relatively slow 10 Mbit line the download time would only increase by 0.01 second. A nice side effect will be that the output will have 'normal' formatting again, which makes debugging a lot easier.

Yes, I just removed str_replace. It is simpler than to write a regular expression.

edited May '18

No, it's not in the code generated by scaffolding. The problem is in the core index.php (front controller) that handles every request. (See Oleg's link to the commit that added str_replace to the project templates.)

That line simply does not exist in Phalcon core/source. Only in DevTools generators.

I have never had such code in my index.php front page controller as I develop my own index page / app structure. So yeah - that's generated by DevTools - the easiest is to just remove it.

edited May '18

Yeah ok... I'm think of that file being in the 'core' of devtools. As in... it's not in some scaffolding code or so. Matter of perspective. :-)

Anyway, should it be removed from the devtools source? I think of it as a bug because some things do not work as they should.

[update] Although the bug is still present, I'm marking the thread as solved, because the Git commit that Oleg linked to answered the questions in my opening post.