How can I determine the best scores to use for the tag, quarantine and block thresholds for the Barracuda filtering?
Incoming Email, Barracuda
The Spam filters that Oracle implements are Barracuda Networks appliances. The method that Barracuda Networks uses to rate Spam uses a point system. Spam messages generally have certain attributes that help to identify them as Spam. Each of these attributes is assigned a point value and the points for all of the applicable attributes for each email are added together to arrive at the final Spam score. It is this Spam score that is compared to the three scores listed on the Preferences > Spam Settings page to determine how the message should be treated -- whether it should be blocked or quarantined.
Default Scoring for Email (as set by Oracle's Hosting group):
The recommended scores that display on the Preferences > Spam Settings page are recommendations written into the Barracuda page itself. However, Oracle's Hosting team has set the default scores to be different than the values recommended on the page. This is because we recommend that the blocking feature be set higher initially.
The email blocking feature is disabled by setting the Block Score to a value of 10. With blocking disabled, you can be sure that emails are not being blocked and your legitimate customer requests are delivered to the mailbox. The default value for this setting is 9.
Note: Even with blocking disabled in Barracuda, email messages may be discarded based on the mailbox configuration or your rule configuration within the Oracle B2C Service application itself.
By default, the quarantine feature is disabled as configured on the Preferences > Quarantine Settings page. This ensures that email messages are delivered to your hosted mailbox and are processed to become incidents based on your mailbox settings within your Oracle B2C Service application.
Oracle's default score are set with a Tag score of 3.5 and a Quarantine score of 10 (disabled). Email messages with a score that exceeds the Quarantine Score will have #QUAR# pre-pended to the Subject line of the incident. Email messages with a score between the Tag Score and Quarantine Score result in incidents with #TAG# pre-pended to the Subject line.
The default scoring as configured in Oracle is:
Tag Score: 3.5
Quarantine Score: 10.0 (disabled)
Block Score: 9.0
Tag Score: If a message has a spam score above the tag value (3.5 by default), the text "#TAG#" is added in the subject line. This is then passed on to your techmail mailbox, and incidents created from these messages will have #TAG# in the subject field. This is used to flag messages as potential SPAM. Within your workflow rules, you can add a rule to route the messages with #TAG# in the subject differently than other incidents.
Quarantine Score: If a message has a cumulative score above the set value the Barracuda appliance will put #QUAR# in the subject. If you have the Quarantine feature enabled, the Barracuda will keep the message in quarantine, and the message will not be delivered to your techmail mailbox. When you first log in to the Barracuda, the first screen you see shows the messages that are quarantined. From this screen, you can delete or deliver the messages that are held in the quarantine.
Block Score: If this is set to a value other than 10, any message with a spam score greater than the block score is not delivered or quarantined. It is instead deleted. You will not see the message in your Support Console and the message will not be visible via the quarantine on the Barracuda. Effectively this message will become permanently lost.
Note: If the block score is lower than the quarantine score, blocking takes precedence and the message is blocked (as opposed to being flagged as quarantined). Similarly, if the quarantine score is less than the tagging score, the quarantine score takes precedence. In this case, no messages will ever get #TAG# put into the subject line (effectively disabling tagging).
Evaluating Individual Email Scores
If the EGW_SAVE_EMAIL_HEADERS configuration setting is enabled in your Oracle B2C Service application, you can view the email header of an incident by clicking on the envelope icon next to the contact name in the incident thread. The envelope only displays if the end-user submits the incident or an update to an incident via email (and not the Ask a Question or Account - Questions page). For more information on this setting, refer to Answer ID 1595: Enabling Email Headers for Incidents.
Within the email header, look for a section similar to the following:
X-Barracuda-Spam-Status: No, SCORE=2.07 using per-user
scores of TAG_LEVEL=3.5 QUARANTINE_LEVEL=9.0
HTML_IMAGE_RATIO_04, THREAD_INDEX, THREAD_TOPIC,
X-Barracuda-Spam-Report: Code version 3.02, rules version 3.0.17279
Rule breakdown below pts rule name description
0.30 THREAD_TOPIC Thread-Topic: ...(Japanese Subject)...
0.12 X_PRIORITY_HIGH Sent with 'X-Priority' set to high
0.30 THREAD_INDEX thread-index:
0.18 HTML_IMAGE_RATIO_04 BODY: HTML has a low ratio of
text to image area
1.17 HTML_ATTR_UNIQUE BODY: HTML appears to have random
attributes in tags
In this case, the score calculated for the incoming email is 2.07. Within the following lines of the header, the TAG_LEVEL, QUARANTINE_LEVEL, and KILL_LEVEL correspond to the Tag, Quarantine, and Block scores configured in the Barracuda. This allows you to determine how the individual email compared to your scores. The section below the dashed lines is a breakout of how the email's spam score was calculated based on the attributes within that email.
Determining Appropriate Scores
You can review the scores of both Spam messages and legitimate messages in order to determine the score values to set in the Barracuda for your mailbox. By evaluating the spam scores for email that should be blocked or quarantined, you can determine the threshold values to use for tagging, quarantining, or blocking incoming email.
You can experiment with the scores by disabling the quarantine feature on the Preferences > Quarantine Settings page (by default, this is disabled), and then adjusting the Tag and Quarantine scores. This allows you to see two levels of scores in the Support Console just by examining the subject line of incoming incidents. During this process, blocking should be disabled (Block score at 10).
Determining the scoring values involves some experimentation. Set the Tag score to be the value that you want to use for the Quarantine score, and set the Quarantine score to be what you want to use for the Block score. With this approach, the Tag score simulates the quarantine action and the Quarantine score simulates the blocking action. Within the Support Console, incidents flagged with #TAG# indicate incidents that should be considered for being quarantined. Incidents with #QUAR# indicate incidents that should be blocked.
Watch your incoming incidents for a couple of days. Review the spam scores for the incidents with a #TAG# or #QUAR# in the subject line to determine how effective the tag and quarantine scores are working. Absolutely no legitimate messages should have #QUAR# in the subject -- since these messages will end up being blocked when blocking is enabled. If any legitimate messages are flagged with #QUAR#, evaluate the spam score for those messages and then raise the quarantine score to be above that value.
Similarly, incidents with #TAG# in the subject indicate incidents that will be quarantined. If too many legitimate messages are getting #TAG# added to the subject line, increase the Tag score a little - in increments of 0.2 or 0.3. For example, if the Tag score starts with a value of 3.5, you could raise it to 3.7 or 3.8 and then if necessary, raise it further.
If several SPAM messages are creating incidents without either #TAG# or #QUAR#, then lower the Tag and/or Quarantine score, so that these incidents would at least be quarantined or even blocked.
When adjusting the scores, you need to find a balance that allows only a few Spam messages into the Support Console without a #TAG# or #QUAR# while keeping legitimate messages that receive #TAG# in the subject to a minimum. Ideally, if possible, 0 legitimate messages should be flagged with #TAG#.
Once you have adjusted your scores, let the system run with that configuration for what you consider an adequate amount of time -- one week, a month, or whatever you are comfortable with. Continue to evaluate the scores to verify that Spam incidents are getting blocked adequately and then no legitimate incidents have a #QUAR# in the subject line.
Then, once you are comfortable with which incidents are being flagged with #QUAR# and #TAG#, you can implement blocking and the quarantine on your site. Set the Quarantine score value to be the value you used for tagging. Then, set the Block score to be the value used for the quarantine. Next, enable the quarantine feature from the Preferences > Quarantine Settings page in the Barracuda. To enable the quarantine, click Yes for the Enable Quarantine field and then click Save Changes in the heading for that section.
You should now receive a minimal amount of Spam while keeping questionable messages on the Barracuda in the quarantine. You can check the quarantine daily or weekly to delete Spam that was quarantined and to deliver any legitimate messages that were quarantined due to the spam score of the email message.
For example: You could start with the following scores as a starting point:
Tag score = 3.5
Quarantine score = 9.0 (with Quarantine disabled)
Block score = 10 (to disable blocking)
By reviewing the incidents coming in, you determine that there are too many Spam messages that are being created with #TAG# instead of #QUAR# and that these should be blocked outright, so you decide to drop the Quarantine score to 8.0. Through additional evaluation, you determine that 3.5 is too low a value for the tag score and that too many legitimate emails would be quarantined, so you test further to determine that the tag score works best if set to 4.2.
Therefore, based on testing, the scores that seem to work best are:
Tag score = 4.2
Quarantine score = 8.0 (with Quarantine disabled)
Block score = 10 (to disable blocking)
After running with this configuration for a time, you determine you are ready to implement blocking and quarantine on your mailbox, so you edit the scores to be as follows:
Tag score = 4.3 (which disables tagging of incidents)
Quarantine score = 4.2
Block score = 8.0 (so blocking is enabled)
Then, you need to enable the quarantine feature on the Preferences > Quarantine Settings page.
Note: To disable tagging (including #TAG# in the subject line of incidents), set the Tag score to be greater than the Quarantine score. If you wish to tag incidents, you can set the Tag score to be less than the Quarantine score. This allows you to further evaluate whether the Quarantine threshold should be modified further.
For more information about how the Barracuda learning system works, refer to Answer ID 9764: Barracuda Bayesian Learning.