Ticket notification emails contain wiki formatting

[chris.alex.thomas@…]

Hi,

when I send an email of a ticket, through the CC list, I find that it sends the wiki formatting as well, since it's a text email, the normal text should be displayed and the wiki formatter should strip out the formatting before the email is sent.

In order to get acceptable results from the ticket display, an example is like thus

1. an item CamelCase
2. another item

but in the email, instead of seeing what I have above, I see this

1. an item with !CamelCase[[br]]
2. another item[[br]]

so of course, people reading the emails, are wondering what all the [[br]] are for and it makes some emails kinda unreadable.

this should be remedied

[anonymous]

sorry jonas, incorrectly assigned the component and forgot my email address from the CC

[cboos]

Yes, a TextFormatter for wiki content would be useful in that case.

As a bonus, the links should be replaced by footnotes, e.g.

 1. an item with CamelCase (1)
 2. another item

----
(1) http://example.org/trac/wiki/CamelCase

[d_dorothy@…]

I came up with a simple patch for now until something fancier could be done (I do not have lots of experience with python regular expressions)

To do this right you need to make a class that inherits from Formatter and knows how to parse through the text using regex. I could not get this to work b/c of my lack of python regex knowledge.

--- notification.py	Wed Nov 29 16:40:02 2006
+++ notification.py	Wed Nov 29 16:34:58 2006
@@ -276,12 +276,42 @@
             self.server.starttls()
             self.server.ehlo()
         if self.user_name:
-            self.server.login(self.user_name, self.password)
+            self.server.login(self.user_name, self.password)
+
+    def removeWikiTags(self, text):
+        import re 
+        #replace [[br]] with new line
+        text = text.replace("[[br]]", "\n")
+        text = text.replace("[[BR]]", "\n")
+        
+        from trac.wiki.api import WikiSystem
+        
+        #return re.sub(WikiSystem(self.env).rules, self.replace, text)
+        
+        #Ignore wiki-macros
+        r = re.compile(r"\[\[.*\]\]")
+        m = r.search(text)
+        while m:
+            if m.end() < len(text):
+                text = text[:m.start()] + text[m.end():]
+            else:
+                text = text[:m.start()]
+            m = r.search(text)
+        
+        #Ignore bold and italic
+        r = re.compile("''+")
+        tmp = r.split(text)
+        text = ""
+        for m in tmp:
+            text += m
+        
+        return text
+        
 
     def send(self, torcpts, ccrcpts, mime_headers={}):
         from email.MIMEText import MIMEText
         from email.Utils import formatdate, formataddr
-        body = self.hdf.render(self.template_name)
+        body = self.removeWikiTags(self.hdf.render(self.template_name))
         projname = self.config.get('project', 'name')
         public_cc = self.config.getbool('notification', 'use_public_cc')
         headers = {}

If this helps let me know.

[d_dorothy@…]

I have made some classes that do this right. Except for tables. Not sure how best to do these and make them readable in plain text. I just left some of the wiki formatting there to help with readability in plain text.

Also my unit test is sort of system dependent. I looked into the TracDev/UnitTests documentation but did not quite understand how to do some of the stuff. (sorry for my limited python understanding, I am working on this)

Also any ideas on where exatly this code should be put? Should it be placed directly in notification.py or should it be a separate file referenced by notification.py?

I think that it will be able to be plugged in similar to my previous post, though I have not tested this.

body = NotifyFormatter(env).format(self.hdf.render(self.template_name))

I will attach my file.

[d_dorothy@…]

proposed NotifyFormatter? class (and related one line class and convenience function)

[chris.alex.thomas@…]

Hi,

isnt the easiest way to solve this problem to simply NOT reformat the mail before it's sent? to simply take the text that gets reformatted by the wiki and sending the email of the data which is input into the wiki reprocessor instead of sending the output of the wiki reprocessor?

I understand that sometimes the simplest way is sometimes the worst design, because perhaps the flow of the code would be broken by "hacking" this to work and sometimes this can only happen whilst maintaining the great design, is not by hacking, but a redesign of that section of code, which has implications for other code as well.

but it sounds the easiest way. Anyone with a clue know whether this idea can fly?

[d_dorothy@…]

I looked into what you asked about. I went further back and tried to track a notification from the beginning.

The ticket change is stored in the db with the wiki syntax in it and the notification mechanism appears to just suck this data up to generate the email header and body. This means that if you do not run the text retrieved from the database through a formatting mechanism you will get the output that is currently seen. For this reason you must do something to format it.

It is possible to just strip out the Wiki formatting (using regular expressions) and return it in plain text but that does not always lend itself to readability in the text format (think about tables, lists, code blocks, etc.).

If I have missed an important detail please point out where I am wrong so I can provide you with better answers.

[eblot]

See also #4516

[trac@…]

Another option is to render it into HTML and send it as a multipart MIME with the HTML (found this ticket because a user [complained to / asked] me today).

[Markus Tacker <m@…>]

Replying to trac@revragnarok.com:

Another option is to render it into HTML and send it as a multipart MIME with the HTML (found this ticket because a user [complained to / asked] me today).

See #2625 for this.

[anonymous]

Replying to d_dorothy@bellsouth.net:

I think that it will be able to be plugged in similar to my previous post, though I have not tested this.

Did anyone tested this? I couldn't make it work with Trac 0.10.4 yet. Or else is there any better workaround for this? I could not locate such a thing on Trac Users.

I will give more debugging details in a separate email.

Sumith

[anonymous]

Replying to anonymous:

I will give more debugging details in a separate email.

This is what I get:

2007-06-13 12:14:16,365 Trac[web_ui] ERROR: Failure sending notification on change to ticket #77: global name 'NotifyFormatter' is not defined
Traceback (most recent call last):
 File "D:\Python24\Lib\site-packages\trac\ticket\web_ui.py", line 562, in _do_save
 tn.notify(ticket, newticket=False, modtime=now)
 File "D:\Python24\Lib\site-packages\trac\ticket\notification.py", line 129, in notify
 NotifyEmail.notify(self, ticket.id, subject)
 File "D:\Python24\lib\site-packages\trac\notification.py", line 216, in notify
 Notify.notify(self, resid)
 File "D:\Python24\lib\site-packages\trac\notification.py", line 115, in notify
 self.send(torcpts, ccrcpts)
 File "D:\Python24\Lib\site-packages\trac\ticket\notification.py", line 275, in send
 NotifyEmail.send(self, torcpts, ccrcpts, hdrs)
 File "D:\Python24\lib\site-packages\trac\notification.py", line 293, in send
 body = NotifyFormatter(env).format(self.hdf.render(self.template_name))
NameError: global name 'NotifyFormatter' is not defined

What I did simply was

1. Copied notifyformatter.py to D:\Python24\lib\site-packages\trac\.
2. Modified D:\Python24\lib\site-packages\trac\notification.py as given in comment:ticket:4270:5

I noticed that .pyc or .pyo files for notifyformatter is not created.

What I have missed here?

Sumith

[cboos]

#7095 was closed as duplicate (it was focusing on the InterWiki links - see comment:2 for how the links could be handled).