Skip to main content

Welcome to Mark Baggett - In Depth Defense

I am the course Author of SANS SEC573 Automating Information Security with Python. Check back frequently for updated tools and articles related to course material.




FREQ and FREQ-SERVER UPDATE

While sitting in SANS SEC511 I listened to @sethmisenar lament the difficulty in using existing tools to detect DGA (Dynamically Generation Algorithm) hostnames used by malware. There are lots of AI based tools out there that do this but some are rather complex. I thought I could quickly write a tool that would work. In about 30 minutes I threw together some old code I had lying around from a SQL Inction tool I worked on and I had a working proof of concept. freq.py was born and it worked pretty well. A year later @securitymapper had me wrap it in a web interface so he could query it from a SIM and then the tool took off. It turns out to be a pretty effective technique and gained some popularity and wide use! This is a rewrite of the tool that incorporates some lessons learned and performance enhancments.
Improvements:
-Only one table is required for case sensitve or insensitive lookups. The tables are all case sensitive. You can turn off and on case sensitivity and the .probability lookups will do what is needed to make them case insensitive.
-Ignored char - Like ignore_case the characters are only ignored in the calculations of the probability. They are not ignored in the building of the table
-Speed. Like I said. It was a proof of concept and never really built with any performance in mind. This fixes that.
-Accuracy. Some errors in calulations were identified by Pepe Berta (thanks!). This fixes those and several others. If you find others let me know.
-Two calculations - I've added a second frequency score that i've calculated differently. It will requires some testing to see if it is more useful than the previous number in detecting random hosts.
Version Compatibility: freq.py and freq_server.py will work in either Python2 or Python3. 

Popular posts from this blog

Awesome Keyboard Tricks - Clevo/Sager Backlight control from Powershell

I'm back on Windows.   After 8 years on a Macintosh I just couldn't go another day with ONLY 16GB of RAM.   I priced it out and for the cost of a top of the line MacBook I could get a tricked out PC with 32GB of ram and 2.5 TB or hard drive space (1.5 of it being SSD).   So I made the switch.  To get a top performing laptop I ended up buying a gaming machine from xoticpc.com.   The model is Sager NP9752 ( Clevo P750ZM ).    I have to say I like it quite a bit.    One of the features I was curious about was the "Programmable backlit keyboard".   With it you can set your keyboard backlight to various colors and light movement patterns.    Now, when I hear "programmable" I think APIs.   I was a little disappointed to find out there weren't any documented APIs that I could use to control the keyboard.    Your only choice is to use their built in tool to configure the lights on the keyboard.   That stinks.  I want to be able to change key colors automatically

SRUM-DUMP and SRUM_DUMP_CSV Ported to Python 3

SRUM_DUMP and SRUM_DUMP_CSV have been ported to Python3 and are available for download from the PYTHON3 branch of my github page. https://github.com/MarkBaggett/srum-dump/tree/python3 In moving to Python3 I also updated the modules that I depend upon to parse and create XLSX files and access the ESE database that contains the SRUM data.  I hope that this will fix the issue that some users have experienced with SRUDB.dat files that create very large spreadsheets.  If it does not please let me know and continue to use SRUM_DUMP_CSV.EXE to avoid the XLSX problem. In moving to Python3 you will find the process to be faster. If you would like to run the tools from source instructions for doing so are in the README on the github page.

New tool Freq_sort.py

I read an article on Fireeye's website the other day where they used Machine Learning to eliminate a lot of the noise that comes out of tools like strings.  It's pretty interesting and looks like it would save me some time when looking through malware. https://www.fireeye.com/blog/threat-research/2019/05/learning-to-rank-strings-output-for-speedier-malware-analysis.html I wondered how effective freq.py scores would be in helping to eliminate the noise.  45 minutes and 29 lines of Python code later I have something that looks like it works.  Check out freq_sort.py. Before freq_sort.py here is the output of strings on a piece of malware: student@573:~/freq$ strings -n 6 malware.exe | head -n 20 !This program cannot be run in DOS mode. e!Rich `.rdata @.data .pdata @.gfids @.rsrc @.reloc \$0u"H L$ SVWH K SVWH |$ H;_ <bt%<xt!<Zt |$ AVH l$ VWAV L$ SUVWH UVWATAUAVAWH 0A_A^A]A\_^] UVWATAUAVAWH @A_A^A]A\_^] After freq_sort.py the useful stings quickly bubble to t

Security Onion getting the most from Freq.py and Domain States

My talk at Security Onion conference has been posted and is available for viewing here.

SRUM DUMP and SRUM DUMP CSV Updated

An issue was reported where is some conditions SRUM_DUMP would stop processing and print the following error to the screen. UnboundLocalError: local variable 'sid_str' referenced before assignment The issue was that sometimes the SRUM database had entries in it that were all zeros. OrderedDict([('IdType', 3), ('IdIndex', 38127), ('IdBlob', '0000000000000000')]) I've released an update that handles the anomoly althought I do not understand the circomstances of why Windows would record all zero's for as the user SID. The issue was fixed and new versions of both SRUM DUMP and SRUM DUMP CSV were released.