| August 2014 |
|
October 2014 | |||||||||||||||||||||||||||||||||||||||||||||||||
pcap go 0 add time 360000 save http session file test.dat save session table test-session.dat set json ## if I want to save the standard json filesTo mask a source IP:
set internal addr-mask <src_ip_internal_address> 255.255.0.0For example
set internal addr-mask 192.168.59.2 255.255.0.0
gdb --args <executable_with_options>Set a break point
br <file>:<line_number>Run
rContinue
cGo to next line
n # note: pressing the return key will run the previous commandJump inside a function call
sPrint some program variable within the scope
p <variable>Quit
q
;; Display line number on the left side
(global-linum-mode t)
I also enabled 'Ido' setting to switch between the buffers and/or files:
;; Enable ido mode
(require 'ido)
(ido-mode t)
(setq ido-enable-flex-matching t) ;; enable fuzzy matching
To switch between buffers, press "C-x b". To open a new file, press "C-x C-f".
To comment a block of code use "alt;" after selecting the block.
spark-shellNow, let's open one of the hadoop files ("hadoop fs -ls")
val data = sc.textFile("myfile.json")
Check how many elements we have in the data and parse it
data.count val pdata = data.map( x => new Parsed(x) )Apply filter to the above parsed data
val out = pdata.filter( x=> isInternal(x.src_ip,"blah") && !isInternal(x.dst_ip,"blah" ))Get an array of the first 10 elements of the filtered data
out.take(10).foreach(x => println(x) )To filter the TCP port (number 8663)
val tcpdata = pdata.filter( x=> x.dst_port == 8663)To filter the Stratum port (number 8332)
val strdata = pdata.filter( x=> x.dst_port == 8332)Sometimes we may need to convert hex characters to ascii for easy reading
print(hexToASCII(""))
hadoop fs -copyFromLocal test.json /user/kalanand/.Run pyspark on the local cluster:
pyspark --master localConvert pcapng files to pcap:
tcpdump -r <pcapng_file> -w <pcap_file>
alt ;(both keys together).
alt %
package example
object hello {
def main(args: Array[String]) =
if (args == null) println("hello world!")
else println("hello world!" + args.toList) //> main: (args: Array[String])Unit
main(null) //> hello world!
main(Array("How", "are", "you ?")) //> hello world!List(How, are, you ?)
}
And the first worksheet:
package greeter
object first_worksheet {
println("Welcome to the Scala worksheet") //> Welcome to the Scala worksheet
val x = 5 //> x : Int = 5
def increase(i: Int) = i + 1 //> increase: (i: Int)Int
increase(x) //> res0: Int = 6
}
Command Shift F(all three keys together). This is sorthand for
Command S
200: OK # Very frequent 201: Created 202: Accepted 204: No Content 205: Reset Content 206: Partial Content 300: Multiple Choices 301: Moved Permanently (redirect) 302: Found (temporary redirect) 303: See Other (temporary redirect) 304: Not Modified # Very frequent 305: Use Proxy 307: Temporary Redirect 400: Bad Request 401: Unauthorized 403: Forbidden 404: Not Found 405: Method Not Allowed 406: Not Acceptable 407: Proxy Authentication Required 408: Request Timeout 409: Conflict 410: Gone 415: Unsupported Media Type 500: Internal Server Error 502: Bad Gateway 503: Service Unavailable 504: Gateway Timeout
A list of valid TLD names (as a text file) can be found at the IANA website
http://data.iana.org/TLD/tlds-alpha-by-domain.txt
A good description of the internet addressing scheme can be found at
http://www.iana.org/numbers
sudo apt-get install python-pip sudo pip install couchdbOften times one stores data notes in such a way that each line is a new JSON. So, in order to parse the information contained in the note one needs to do
import json
data = []
with open('file') as f:
for line in f:
data.append(json.loads(line))
Colloquy is an IRC chat client for Mac. For details and download, see
http://colloquy.info.
September 23rd
Scala week-1: simple algebraic stuff like root finding using Newton's method
package week1
object session {
1 + 3 //> res0: Int(4) = 4
def abs(x: Double) = if (x < 0) -x else x //> abs: (x: Double)Double
def sqrt(x: Double) = {
def sqrtIter(guess: Double): Double =
if (isGoodEnough(guess)) guess
else sqrtIter(improve(guess))
def isGoodEnough(guess: Double) = abs(guess - x / guess) < 0.001
def improve(guess: Double) = (guess + x / guess) / 2
sqrtIter(1.0)
} //> sqrt: (x: Double)Double
sqrt(2) //> res1: Double = 1.4142156862745097
sqrt(4) //> res2: Double = 2.0000000929222947
sqrt(0.001) //> res3: Double = 0.031642015868650786
sqrt(0.1e-20) //> res4: Double = 9.765625000000002E-4
sqrt(1.0e20) //> res5: Double = 1.0E10
sqrt(1.0e50) //> res6: Double = 1.0E25
def gcd(a: Int, b: Int): Int =
if (b == 0) a else gcd(b, a % b) //> gcd: (a: Int, b: Int)Int
gcd(14, 21) //> res7: Int = 7
def factorial(n: Int): Int =
if (n == 0) 1 else n * factorial(n - 1) //> factorial: (n: Int)Int
factorial(4) //> res8: Int = 24
September 24th
Git rehash
Clone a new package from source
git clone <source_code_location_with_path.git>
Checkout an existing branch from the source repository
git checkout <some_existing_branch_I_need_to_checkout>
To see which branch I am currently working in
git branch
To fetch branches and/or tags from the source repository along with the objects necessary to complete their histories
git fetch
To incorporates changes from the remote repository into the current branch
git pull
To check modification/commit status of the tracked files only
git status -uno
To track a new file or to prepare to commit a modified file
git add <file>
To commit (make sure to include JIRA issue number in the comment, otherwise
the remote repository will decline push request)
git commit -m <comment>
To push this commit to the remote repository
git push
To create a totally new branch and switch to this new branch in one step
git checkout -b <new_branch_name>
To create a totally new branch while still remaining on the current branch
git branch <new_branch_name>
To merge some other branch ("other_branch") into the current branch ("this_branch")
git fetch
git checkout <that_branch>
git pull
git checkout <this_branch>
git pull
git merge <that_branch>
September 26th
Scala week-1 continued: map-reduce, inner product, ...
package week1
object exercise {
def factorial(n: Int): Int = {
def loop(acc: Int, n: Int): Int =
if (n == 0) acc
else loop(acc * n, n - 1)
loop(1, n)
} //> factorial: (n: Int)Int
factorial(4) //> res0: Int = 24
def product(f: Int => Int)(a: Int, b: Int): Int =
if (a > b) 1 else f(a) * product(f)(a + 1, b) //> product: (f: Int => Int)(a: Int, b: Int)Int
product(x => x * x)(3, 4) //> res1: Int = 144
def fact(n: Int) = product(x => x)(1, n) //> fact: (n: Int)Int
fact(5) //> res2: Int = 120
def mapReduce(f: Int => Int, combine: (Int, Int) => Int, zero: Int)(a: Int, b: Int): Int =
if (a > b) zero else
combine(f(a), mapReduce(f, combine, zero)(a + 1, b))
//> mapReduce: (f: Int => Int, combine: (Int, Int) => Int, zero: Int)(a: Int, b:
//| Int)Int
def product1(f: Int => Int)(a: Int, b: Int): Int =
mapReduce(f, (x, y) => x * y, 1)(a, b) //> product1: (f: Int => Int)(a: Int, b: Int)Int
def fact1(n: Int) = product1(x => x)(1, n) //> fact1: (n: Int)Int
fact1(5) //> res3: Int = 120
}
September 30th
We can filter a pcap file and dump the output to a new pcap.
The command is:
tcpdump -r <input.pcap> -w <output.pcap> <filter_string>
For example:
tcpdump -r zeroaccess.pcap -w new.pcap "tcp and port 80 and host 192.168.58.2 and (tcp[((tcp[12:1]&0xf0)>>2):4]=0x47455420 or (tcp[((tcp[12:1]&0xf0)>>2):4]=0x48545450 and tcp[((tcp[12:1]&0xf0)>>2)+4:2]=0x2f31))"
Go to August's log
Last modified: Tue Sep 30 22:15:56 PDT 2014