URIBL: fail, very strong obfuscated IP found in URI

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

URIBL: fail, very strong obfuscated IP found in URI

K Post
In reviewing our block reports, I'm seeing a handful of legitimate mail being rejected due to:
URIBL: fail, very strong obfuscated IP found in URI

I don't know enough about this technique to know for sure, but these messages all seem to have URL's in them with tracking codes (I assume) built in and I think that's what's triggering the erroneous catch.  For example:

thesender.com (example here) is a the domain of a good company that the recipient does business with and the email is legitimate.

I just disabled URIBLNoObfuscated to stop this problem, but we would like to disable obfuscated hostnames or ip's.

I don't know if this is a good idea, but could the detection code be tweaked to ignore "very strong" obfuscation unless it's in the hostname part of a URL?  For example:

http://0x9A3F0800CEBF9E37/subpage/page will FAIL

As always, thanks.

------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today.http://sdm.link/intel
_______________________________________________
Assp-test mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/assp-test
Reply | Threaded
Open this post in threaded view
|

Re: URIBL: fail, very strong obfuscated IP found in URI

Thomas Eckardt/eck
>For example:

>http://www.sender.com/0x9A3F0800CEBF9E37 will PASS but
>http://0x9A3F0800CEBF9E37/subpage/page will FAIL

If it would be sooooo.... simple!

This is the used regular expression:

(?:
(?:(?i:[\=\%][46]8|\&\#(?:0?72|104)\;?|h)
(?i:[\=\%][57]4|\&\#(?:0?84|116)\;?|t)
|(?i:[\=\%][46]6|\&\#(?:0?70|102)\;?|f))
(?i:[\=\%][57]4|\&\#(?:0?84|116)\;?|t)
(?i:[\=\%][57]0|\&\#(?:0?80|112)\;?|p)
(?i:[\=\%][57]3|\&\#(?:0?83|115)\;?|s)?
|
(?:
(?i:[\=\%][57]0|\&\#(?:0?80|112)\;?|p)
(?i:[\=\%][57]5|\&\#(?:0?85|117)\;?|u)
(?i:[\=\%][57]2|\&\#(?:0?82|114)\;?|r)
(?i:[\=\%][57]2|\&\#(?:0?82|114)\;?|r)
(?i:[\=\%][57]3|\&\#(?:0?83|115)\;?|s)?
)
)
(?:[\=\%]3[aA]|\&\#0?58\;?|\:)
(?:[\=\%]2[fF]|\&\#0?47\;?|\/){2}
(?:[^\@]*?(?:\@|[=%]40|\&\#0?64\;?))?
(?:(?:(?:=|\%|(?:0|\&\#)[xX])[0-9a-fA-F]+|(?:\&\#|\\)?\d+);?)
(?:(?:[\=\%]2[eE]|\&\#0?46\;?|\.)
|
(?:[\=\%]3[aA]|\&\#0?58\;?|\:){1,2})?)+[^\.\w\@]

I think I found the reason for the match - and the next release will try to fix this.
BUT for me , this URI looks very obfuscated - why?

1. it uses an unknown protocol parameter in a GET value (s=) : gusqr://
2. it uses an unknown hostname there: qkvr.hnpfmd.dnn
3 in t= it uses: 2@031-040 - a username followed by an @ and an octal number range

For me this looks like: "if you have my trojan available - nice, I'll start it now"
This gives me the idea, to make this check much more restrictive in future. So you'll have the problem again in some time.

Thomas





Von:        K Post <[hidden email]>
An:        ASSP development mailing list <[hidden email]>
Datum:        23.12.2016 16:16
Betreff:        [Assp-test] URIBL: fail, very strong obfuscated IP found in URI




In reviewing our block reports, I'm seeing a handful of legitimate mail being rejected due to:
URIBL: fail, very strong obfuscated IP found in URI


I don't know enough about this technique to know for sure, but these messages all seem to have URL's in them with tracking codes (I assume) built in and I think that's what's triggering the erroneous catch.  For example:
http://etrack.thesender.com/t/ccbbaLT6QADKsHAuHEfaBFQA1CHQK42aaaa?t=2@031-040&f=esdfcpmgdseobebbbm_jx.Zocinscem.dnn&k=C2w&w=&s=gusqr://qkvr.hnpfmd.dnn/+esdccpmgdseerobfbbkm/opr1r

thesender.com (example here) is a the domain of a good company that the recipient does business with and the email is legitimate.

I just disabled URIBLNoObfuscated to stop this problem, but we would like to disable obfuscated hostnames or ip's.

I don't know if this is a good idea, but could the detection code be tweaked to ignore "very strong" obfuscation unless it's in the hostname part of a URL?  For example:

http://www.sender.com/0x9A3F0800CEBF9E37 will PASS but
http://0x9A3F0800CEBF9E37/subpage/page will FAIL

As always, thanks.------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today.http://sdm.link/intel_______________________________________________
Assp-test mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/assp-test




DISCLAIMER:
*******************************************************
This email and any files transmitted with it may be confidential, legally privileged and protected in law and are intended solely for the use of the
individual to whom it is addressed.
This email was multiple times scanned for viruses. There should be no known virus in this email!
*******************************************************


------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today.http://sdm.link/intel
_______________________________________________
Assp-test mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/assp-test
Reply | Threaded
Open this post in threaded view
|

Re: URIBL: fail, very strong obfuscated IP found in URI

K Post
I didn't mean to imply that any of this was easy.  If it was, I'd do it myself and give you the code.  I barely understand what we're protecting against here, I just know that we're seeing a lot of false positives here.

That's one heck of a regex!!  Would it be unsafe to only compare it to hostnames?  Are you seeing spam (or phishing attacks) that have clear hostnames about obfuscated file/query string?  I haven't, but I don't have the depth of knowledge, experience, or exposure that you do.  I'm curious to know what we're trying to avoid blocking more than just the hostname and if it's "worth it" vs false positives.

>.1. it uses an unknown protocol parameter in a GET value (s=) : gusqr:// 
>2. it uses an unknown hostname there: qkvr.hnpfmd.dnn 
1 & 2 are in the query string.  Do we care?  Aren't we most interested in looking for obfuscated hostnames and file paths?
I'm curious here, not being argumentative: what's the spammer strategy that we're protecting against by rejecting these messages?

>3 in t= it uses: 2@031-040 - a username followed by an @ and an octal number range 
 don't tons of tracking images use similar methodology?  

>For me this looks like: "if you have my trojan available - nice, I'll start it now" 
Well that's scary - but don't plenty of legitimate emails do the same thing where there's extra stuff in the query string?

>This gives me the idea, to make this check much more restrictive in future. So you'll have the problem again in some time. 
Would love to know your thinking - 

I worry when we restrictive to absolutely be able to block a certain type of email when there are lots of false positives caused by the restriction.  It's not like we're going to be able to change the way that legitimate emailer send messages (especially with tracking codes in URL's) and we can't (or at least I can't) simply say to our users that they're not allowed to get those mails because they have a way of putting URLs that some phishers might.  It's all about that delicate balance between stopping the spams/scams and insuring deliverability of legitimate messages.

HAPPY HOLIDAYS!




------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today.http://sdm.link/intel
_______________________________________________
Assp-test mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/assp-test
Reply | Threaded
Open this post in threaded view
|

Re: URIBL: fail, very strong obfuscated IP found in URI

Thomas Eckardt/eck
>I barely understand what we're protecting against here
Yes.

http://www.pc-help.org/obscure.htm

>1 & 2 are in the query string.  Do we care? 
Yes. They may contain redirects.

Thomas


Von:        K Post <[hidden email]>
An:        ASSP development mailing list <[hidden email]>
Datum:        24.12.2016 19:38
Betreff:        Re: [Assp-test] URIBL: fail, very strong obfuscated IP found in URI




I didn't mean to imply that any of this was easy.  If it was, I'd do it myself and give you the code.  I barely understand what we're protecting against here, I just know that we're seeing a lot of false positives here.

That's one heck of a regex!!  Would it be unsafe to only compare it to hostnames?  Are you seeing spam (or phishing attacks) that have clear hostnames about obfuscated file/query string?  I haven't, but I don't have the depth of knowledge, experience, or exposure that you do.  I'm curious to know what we're trying to avoid blocking more than just the hostname and if it's "worth it" vs false positives.

>.1. it uses an unknown protocol parameter in a GET value (s=) : gusqr:// 
>2. it uses an unknown hostname there: 
qkvr.hnpfmd.dnn 
1 & 2 are in the query string.  Do we care?  Aren't we most interested in looking for obfuscated hostnames and file paths?
I'm curious here, not being argumentative: what's the spammer strategy that we're protecting against by rejecting these messages?

>3 in t= it uses: 2@031-040 - a username followed by an @ and an octal number range 
 don't tons of tracking images use similar methodology?  

>For me this looks like: "if you have my trojan available - nice, I'll start it now" 
Well that's scary - but don't plenty of legitimate emails do the same thing where there's extra stuff in the query string?

>This gives me the idea, to make this check much more restrictive in future. So you'll have the problem again in some time. 
Would love to know your thinking - 

I worry when we restrictive to absolutely be able to block a certain type of email when there are lots of false positives caused by the restriction.  It's not like we're going to be able to change the way that legitimate emailer send messages (especially with tracking codes in URL's) and we can't (or at least I can't) simply say to our users that they're not allowed to get those mails because they have a way of putting URLs that some phishers might.  It's all about that delicate balance between stopping the spams/scams and insuring deliverability of legitimate messages.

HAPPY HOLIDAYS!


------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today.http://sdm.link/intel_______________________________________________
Assp-test mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/assp-test




DISCLAIMER:
*******************************************************
This email and any files transmitted with it may be confidential, legally privileged and protected in law and are intended solely for the use of the
individual to whom it is addressed.
This email was multiple times scanned for viruses. There should be no known virus in this email!
*******************************************************


------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today.http://sdm.link/intel
_______________________________________________
Assp-test mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/assp-test
Reply | Threaded
Open this post in threaded view
|

Re: URIBL: fail, very strong obfuscated IP found in URI

Thomas Eckardt/eck
In reply to this post by K Post
2.5.4 16359 should address this.

Thomas





Von:        K Post <[hidden email]>
An:        ASSP development mailing list <[hidden email]>
Datum:        24.12.2016 19:38
Betreff:        Re: [Assp-test] URIBL: fail, very strong obfuscated IP found in URI




I didn't mean to imply that any of this was easy.  If it was, I'd do it myself and give you the code.  I barely understand what we're protecting against here, I just know that we're seeing a lot of false positives here.

That's one heck of a regex!!  Would it be unsafe to only compare it to hostnames?  Are you seeing spam (or phishing attacks) that have clear hostnames about obfuscated file/query string?  I haven't, but I don't have the depth of knowledge, experience, or exposure that you do.  I'm curious to know what we're trying to avoid blocking more than just the hostname and if it's "worth it" vs false positives.

>.1. it uses an unknown protocol parameter in a GET value (s=) : gusqr:// 
>2. it uses an unknown hostname there: 
qkvr.hnpfmd.dnn 
1 & 2 are in the query string.  Do we care?  Aren't we most interested in looking for obfuscated hostnames and file paths?
I'm curious here, not being argumentative: what's the spammer strategy that we're protecting against by rejecting these messages?

>3 in t= it uses: 2@031-040 - a username followed by an @ and an octal number range 
 don't tons of tracking images use similar methodology?  

>For me this looks like: "if you have my trojan available - nice, I'll start it now" 
Well that's scary - but don't plenty of legitimate emails do the same thing where there's extra stuff in the query string?

>This gives me the idea, to make this check much more restrictive in future. So you'll have the problem again in some time. 
Would love to know your thinking - 

I worry when we restrictive to absolutely be able to block a certain type of email when there are lots of false positives caused by the restriction.  It's not like we're going to be able to change the way that legitimate emailer send messages (especially with tracking codes in URL's) and we can't (or at least I can't) simply say to our users that they're not allowed to get those mails because they have a way of putting URLs that some phishers might.  It's all about that delicate balance between stopping the spams/scams and insuring deliverability of legitimate messages.

HAPPY HOLIDAYS!


------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today.http://sdm.link/intel_______________________________________________
Assp-test mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/assp-test




DISCLAIMER:
*******************************************************
This email and any files transmitted with it may be confidential, legally privileged and protected in law and are intended solely for the use of the
individual to whom it is addressed.
This email was multiple times scanned for viruses. There should be no known virus in this email!
*******************************************************


------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today.http://sdm.link/intel
_______________________________________________
Assp-test mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/assp-test