White-listing Tor in CloudFlare with PowerShell

CloudFlare has been one of those technologies that has dramatically improved the Internet since its arrival. CloudFlare provides a number of amazing features, from DDOS protection, traffic analytics, performance optimizations and even free SSL! To top it all off, CloudFlare off the majority of their features for an amazing price – FREE!

Unfortunately, there are a group of users who have suffered with the rapid adaption of websites being protected by CloudFlare; Tor users. Whilst the majority of us receive the benefits of CloudFlare, it has made life for Tor users extremely miserable.

What is the issue with CloudFlare and Tor?

To understand the issues CloudFlare and Tor, we first need to understand how CloudFlare protects a website. When you visit a protected website, CloudFlare performs a risk assessment of your IP address. If CloudFlare decides that you are friendly, then you get to view the website. If it decides you are suspicious, then you will either be prompted with a CAPTCHA request or, in the extreme cases, it will deny access to the website.

The Tor is an interesting network, developed with the aim to protect privacy and defend against surveillance; the network has become a bit of a hive of scum and villainy. Due to the high use of Tor for malicious activity, CloudFlare is naturally suspicious of any connections coming from a Tor exit node, which results in a high risk assessment for all Tor users accessing protected websites.

A high risk assessment results in lots of CAPTCHA requests for Tor users to complete. Every protected page that a Tor user visits, will result in a CAPTCHA request, whilst most of us don’t mind completing these, consider how you would feel answering one every 15 to 20 minutes. It impacts productivity and your sanity.

What is the solution?

The trust of the matter is that there isn’t a simple solution, there isn’t a magic “trust Tor” button in the CloudFlare. It is down to individual website operators to white-list the Tor exit IP addresses; this isn’t trivial, requiring operators to maintain a white-list as the Tor network changes, adding new exit IP addresses that join the Tor network, whilst removing those that leave.

Now is the best time to point out the risks of white-listing. White-listing isn’t without its own risks! When you white-list an IP address in CloudFlare, you are telling it to ignore how risky the traffic coming from that address might be. Not all Tor users legitimately want to view the content of your website; some Tor users are malicious, they are out for blood. White-listing significantly changes the security protection of your site, and you need to be extremely aware of the risks it introduces.

Set and Forget White-Listing

Donncha O'Cearbhaill developed a Python based script with the aim of providing a “set and forget” mechanism for white-listing Tor exit IPs. Donncha’s Automatic CloudFlare Tor Exit Whitelister takes your CloudFlare API token and email address and does all the hard work. The script will grab the latest listing of the Tor exit IPs, and then updates CloudFlare’s white list as appropriate.

When I saw the script, my immediate reaction was “what an amazing idea”, my next was, “how could this be made more accessible to more website operators?

Turning to PowerShell

The obvious answer to encouraging more web site operators to white-list Tor exit nodes is to provide the process in more accessible formats. For Windows users, the best option is to provide the process as a PowerShell script.

So I sat down, learn some Python, and then worked on developing a PowerShell script with similar functionality as Donncha’s. This was a super fun and interesting process. In the end, I deliberately tried to stick a similar logic flow, similar parameters are supported (via parameter liases), and the scripts default output is similar. I added some additional verbose and debug options and added progress bars for long operations. I haven’t included support for specifying the CloudFlare API token and email address via environment variables.

The final result can be found over at GitHub, at https://github.com/poshsecurity/PowerShell-CloudFlare-Tor-Whitelist. All of the functionality is provided by the Set-CloudFlareWhiteList.ps1 script.

Please feel free to raise pull requests, issues, or provide any feedback that you may have.

Usage

I have included comment based help, with examples on how to make use of the script. There are three common usage methods.

1. Specifying a CloudFlare token and email. This will white-list exit node IP addresses across all of the domains in your CloudFlare account
2. Specifying a CloudFlare token, email and a DNS zone. This will white-list exit node IP addresses for a specific DNS zone.
3. Specify a Token, email and optionally a DNS zone, and the -ClearFlags parameter. Remove all Tor specific rules for all domains in your CloudFlare account, or for a specific DNS Zone.

Finishing Up

I want to thank Donncha for his work in developing the original Python script. Without his work, I wouldn’t have been able to develop mine.

If you want to find out more about CloudFlare, Troy Hunt has put together a terrific PluralSight course, Getting Started with CloudFlare™ Security. He covers the basics including migration, SSL configuration and even HSTS.

Kieran Jacobsen

Enabling Mobile Device Management with Office 365

Microsoft recently announced that they would be including a Mobile Device Management (MDM) platform as part of Office 365. What this means is that organisations, both small and large now have an extremely easy and powerful MDM available to them, without any additional charge to their Office 365 licencing. Administrators can manage Android, iOS and Windows Phone devices, and enforce various corporate policies and standards.

To support the new MDM functionality, you will need to create two new DNS records in each of your Office 365 domains. I have updated the Posh-Office365CloudFlare script to support the creation of these two additional records. You can create these records via the -MDMEnable parameter.

The process for creating the entries is as simple as:

Register-Office365.ps1 -CloudFlareApiToken <token> -CloudFlareEmailAddress <email> -Domain <domain> -MDMEnable

Just a quick note, I am yet to fully test out the new MDM functionality as none of my existing tenants have enabled for it yet. 

Kieran Jacobsen

Automating Office 365 deployments in CloudFlare

A few weeks ago, I wrote about Posh-CloudFlare, a PowerShell module I created for managing CloudFlare hosted domains. Since then, I was working on extending an Office 365 deployment, and realized that what was needed was a script which could automate the configuration of new domains. With that in mind, I developed a new PowerShell script,  Posh-Office365CloudFlare.

Let's understand the process for the addition and configuration of a new domain for Office 365.

The process starts with the Office 365 Portal. We navigate to the Domains section, click the "Add Domain" button, and after ignoring the introduction, we proceed to step 1. This step starts with us entering our domain name, let's use our old favorite contoso.com. Now we will be asked to verify that we own this domain, either through the creation of a TXT record or an MX record. The typical method is to use is that of a TXT record, created at the root of our desired domain with a value something like "MS=mx********".

After we create the domain, and the wizard successfully sees the appropriate record, we will be allowed to proceed to the next step. Step 2 isn't one that I usually make use of. I typically don't want to modify my users email domains, nor do I want to add new users at this time. I skip this step and move straight on to step 3.

Step 3 starts with another quick introduction screen, and then we will be asked if we would like the DNS for this domain to be managed by Microsoft. Obviously, we are going to answer no and move on. Finally, we reach an important step, we are asked what we want to do with this domain. First, "Outlook for email, calendar, and contacts", or in other words, email; the second, "Lync for instant messaging and online meetings", which is kind of obvious.

If you select “Outlook for email, calendar, and contacts”, then we will be told to create the following records in contoso.com:

• MX - @.contoso.com - which points to contoso-com.mail.protection.outlook.com (priority 0)
• CNAME – autodiscover.contoso.com – which points to autodiscover.outlook.com
• CNAME – msoid.contoso.com – which points to clientconfig.microsoftonline-p.net
• TXT – @.contoso.com – which contains a SPF record

If you select “Lync for instant messaging and online meetings”, then we will need to create the following records for contoso.com:

• CNAME - sip.contoso.com - which points to sipdir.online.lync.com
• CNAME - lyncdiscover.contoso.com - which points to webdir.online.lync.com
• CNAME – msoid.contoso.com – which points to clientconfig.microsoftonline-p.net
• SRV - _sip._tls.contoso.com - with its appropriate port, weight, priority and target
• SRV - _sipfederationtls._tcp.contoso.com - with its appropriate port, weight, priority and target

Reviewing this list of records, we will notice that the only record that changes for each domain is the MX record. The record consists of the domain name we want to add, with dashes replacing the original dots in the domain name. As you can see in the above example, cotoso.com's MX record points to contoso-com.mail.protection.outlook.com, where as awesomecompany.net would point to awesomecompany-net.mail.protection.com. 

What about some records that could actually help our users? What if I said we could redirect sub domains of our own to the Outlook Web Access page? Wouldn't it be awesome if a user entered https://mail.contoso.com into their browser, and ended up with the Outlook Web Access? This can be achieved by creating a CNAME record that points to mail.office.com. Let's have our script create entries for mail and webmail perform this redirection.

Now back to the script.

This was a simple script, it doesn't have any complex logic, it will need the following information:

• CloudFlare API Token and email address; this is obvious as we need to talk to the CloudFlare Client API.
• The domain name.
• Do we want to create mail records? Lync records or both?

This is a very, very simple script, we just need to have a set of New-CFDNSRecord calls, with various controls depending on what we require.

For example, creating the MX record is as simple as:

This script only took an hour or so for testing and development time, however there was quite a bit of effort directed to changes in the Posh-CloudFlare and the New-CFDNSRecord CMDLet. If you look at the diff's between the last few versions, you will notice the following changes:

• The CMDLet now accepts input from the pipeline (in this case via property name).
• Restructure the CMDLet into Begin/Process/End (required for proper handling of pipeline input).
• Implementation of parameter sets.
• Cleanup of the validation of parameters.

I added parameter sets to New-CFDNSRecord with the aim to remove the somewhat faulty validation that I had previously. Whilst this sounded, and looked like it was simple, it actually took a few tried to ensure that the CMDLet would function appropriately. This was really interesting and deserves its own post in the future.

Parameter validation was updated in all of the CMDLets to improve email address validation. Previously, validation consisted of testing for an "@" character. Now I am using a regular expression.

Finally, I have spent some time cleaning up the code, not just within New-CFDNSRecord, but across all of the CMDLets. I have been trying, where possible to use ISE Steroids to ensure that everything I right is neat and presentable; it is a fantastic resource.

My final thought on all of this journey is, why couldn't Microsoft have implemented something like this? Microsoft has integrated the process with a bunch of other DNS providers, including the likes of GoDaddy, Network Solutions, 1 and 1 and even Yahoo Small Business. Why can't it also look at CloudFlare?

You can find the finished script over at GitHub, at Posh-Office365CloudFlare, the script is called Register-Office365.ps1. I have included comment based help with examples.

Kieran Jacobsen

Posh-CloudFlare managing CloudFlare using PowerShell

The aim of the Posh-CloudFlare module is to simply and automate the management of CloudFlare hosted DNS zones using PowerShell and the CloudFlare Client API. I have made the module available via the PoshSecurity GitHub, here Posh-CloudFlare.

I started looking at CloudFlares API several months ago, as part of another post which I am still working on. Back then I was simply looking at the creation and deletion or records.

Things changed when I found that I needed to spend quite a bit of time working with DNS. Provisioning new infrastructure within cloud environments is something I spend a significant amount of time doing, and am actively investigating the automation of it, and as such, become interested in other parts of the API.

This module now implements all of the Client API, with 22 CMDLets in total. To simplify things, I have documented what CMDLet maps to what API call below:

CMDLets	API Actions
get-CFDNSZoneStatistics	3.1 - "stats" - Retrieve domain statistics for a given time frame
get-CFDNSZone	3.2 - "zone_load_multi" - Retrieve the list of domains
get-CFDNSRecord	3.3 - "rec_load_all" - Retrieve DNS Records of a given domain
get-CFDNSZoneStatus	3.4 - "zone_check" - Checks for active zones and returns their corresponding zids
Get-CFIPThreatScore	3.6 - "ip_lkup" - Check threat score for a given IP
get-CFDNSZoneSettings	3.7 - "zone_settings" - List all current setting values
Set-CFDNSZoneSecurityLevel	4.1 - "sec_lvl" - Set the security level
Set-CFDNSZoneCacheLevel	4.2 - "cache_lvl" - Set the cache level
Set-CFDNSZoneDevMode	4.3 - "devmode" - Toggling Development Mode
Clear-CFDNSZoneCache	4.4 - "fpurge_ts" -- Clear CloudFlare's cache
Clear-CFDNSZoneFileCache	4.5 - "zone_file_purge" -- Purge a single file in CloudFlare's cache
Add-CFBlackListIP Add-CFWhiteListIP Remove-CFListIP	4.6 - "wl" / "ban" / "nul" -- Whitelist/Blacklist/Unlist IPs
Set-CFDNSZoneIPVersion	4.7 - "ipv46" -- Toggle IPv6 support
Set-CFDNSZoneRocketLoader	4.8 - "async" -- Set Rocket Loader
Set-CFDNSZoneMinification	4.9 - "minify" -- Set Minification
Set-CFDNSZoneMirage2	4.10 - "mirage2" -- Set Mirage2
New-CFDNSRecord	5.1 - "rec_new" -- Add a DNS record
Update-CFDNSRecord	5.2 - "rec_edit" -- Edit a DNS record
Remove-CFDNSRecord	5.3 - "rec_delete" -- Delete a DNS record

The Client API can be a little tricky at first, I have developed the CMDLets in a manner to simplify the learning curve. Typically any API call which modifies or removes a DNS record, would require a rec_id to be specified. This field can be found by querying all of the records in the zone. I have simplified things by performing the search and other API queries for you. You can still specify a rec_id if you like.

Switches and parameter validation sets have been used to simplify some of the other CMDLets, particularly those around minification, security and other zone wide settings.

Finally I have tried where possible to make good use of the Pipeline. There are still a number of areas that could be improved.

Getting Started

The first thing you will need to do, is obtain your API Token. This can be found on your Account page. You will need this, and the email address you use to sign into CloudFlare for the majority of the CMDLets. For CMDLets which modify DNS Zones or records, you will need to specify the zone as well.

To obtain the module, simply perform a git clone to your preferred module location as below:

I have included a demo script, Posh-CloudFlare-Demo.ps1 at the root level of the module, which you can run on the namespace of your choice. I recommend not using your corporate production domain. At the top of this script, simply update the API Token, Email and domain name fields as required.

You can then run the script, and see it manipulate the DNS zone. I am not responsible if this breaks production. This script shows you each CMDLet and it's output. I don't recommend simply running the script, I recommend stepping through each line so you gain more of an understanding.

Potential Uses

The automatic provisionment of cloud hosted environments is why this was developed as well as another project I will announce in the coming future. For now, I see myself working on at least one module to support the automation of Office 365 provisioning, including creating the TXT, MX and SRV required.

Warnings

Firstly, I haven’t finished up the PowerShell help – Naughty! I will work on this one as I go.

Secondly, there might be some bugs. Whilst I have tried to test the majority of the permutations of the code, I can’t be fully sure I haven’t missed something. If you find one, please feel free to contact me and I will make the required fixes, or even better, push your updates up to GitHub.

Kieran Jacobsen