Splunk Enterprise Software

What is Splunk?

Splunk Enterprise Software (“Splunk”) is a powerful tool for searching and exploring data.

Splunk is often used by system administrators, network administrators, and security gurus, but its use is not restricted to these audiences. There is a great deal of business value hidden away in corporate data that Splunk can liberate.

Who uses Splunk?

Splunk is a powerful platform for analyzing machine data, data that machines emit in great volumes but which is seldom used effectively. Machine data is already important in the world of technology and is becoming increasingly important in the world of business.

It’s used by:

  • Security offices
  • Marketing departments
  • System administrators
  • Network administrators
  • Application development teams
  • Application support teams

splunk_usage

How does it work?

  • Splunk begins with indexing, which means gathering all the data from diverse locations and combining it into centralized indexes.
  • Using the indexes, Splunk can quickly search the logs from all servers and hone in on when the problem occurred.
  • Splunk can then drill down into the time period when the problem first occurred to determine its root cause. Alerts can then be created to head the issue off in the future.

Splunk provides one repository, data indexing, search & visualization for your data.

What are the potential Splunk data sources?

During indexing, Splunk can read machine data from any number of sources. The most common input sources are:

  • files: Splunk can monitor specific files or directories. If data is added to a file or a new file is added to a monitored directory, Splunk reads that data.
  • the network: Splunk can listen on TCP or UDP ports, reading any data sent.
  • scripted inputs: Splunk can read the machine data output by programs

or scripts, such as a Unix® command or a custom script that monitors sensors.
Technically speaking, retrieved events from your indexes are called “events.” If those events are transformed or summarized so that there is no longer a one-tone mapping with events on disk, they are properly called “results.”

Announcements at the Splunk 2015 Conference in a slide

splunk_news

SPL (Search Processing Language)

The Search Processing Language encompasses all the search commands and their functions, arguments and clauses. Search commands tell Splunk Enterprise what to do to the events you retrieved from the indexes. For example, you need to use a command to filter unwanted information, extract more information, evaluate new fields, calculate statistics, reorder your results, or create a chart.

Some search commands have functions and arguments associated with them. Use these functions and their arguments to specify how the commands act on your results and/or which fields they act upon. For example, use functions to format the data in a chart, describe what kind of statistics to calculate, and specify what fields to evaluate. Some commands also use clauses to specify how to group your search results.

There are four broad categorizations for all the search commands: distributable streaming, stateful streaming, transforming, generating.

splunk_spl

Where can I find more information?

http://docs.splunk.com
http://docs.splunk.com/Documentation/Splunk
http://splunkbase.com
http://docs.splunk.com/images/a/a3/Splunk_4.x_cheatsheet.pdf
http://answers.splunk.com/
Reference Guide http://www.splunk.com/web_assets/pdfs/secure/Splunk_Quick_Reference_Guide.pdf
Educational Videos http://www.splunk.com/view/education-videos/SP-CAAAGB6
Splunk Blogs http://blogs.splunk.com/
Splunk Wiki http://wiki.splunk.com/Special:SplunkSearch/wiki?q=your-query
Certification and education http://www.splunk.com/view/education/SP-CAAAAH9

Print Friendly

The HoloLens and Holographic Academy Experience

I consider myself extremely lucky, part of the future, special and being in the right place at the right time.

This is the HoloLens website.

As part of attending the Microsoft Build conference I got a chance to attend the Holographic Academy. I was one of the first few hundred developers to try developing an app for HoloLens. It was four and a half quality hours in a hotel hall with a big Microsoft team and magic all around. No cameras or recording devices allowed- so I don’t have photos to share. It’s still a lot of secrecy and unanswered questions but what a spectacular unveiling of the technology of the future!

Personal Setting
Somebody came with an optician tool to measure something about my eyes. I don’t understand it all but I believe it measures the space between the pupils. I got 60 as a number and I think it was the smallest number from the people around me- go figure.

The HoloLens customizations and deployments can be done wirelessly but in our with unreliable network, we had to connect the HoloLens to the computer through an USB cable. As a first step, I needed to access the device through the browser with an IP address & port number. You access a website where you plug in the number to customize your HaloLens.

The Development Tools
The tools you need are Unity for the 3D work (https://unity3d.com/) and Visual Studio 2015 for the compilation of the code and deployment.

I opened an existent application, built it in Unity and compiled/deployed it from Visual Studio. When compiling it, I had to look at the spot where I expected to see my hologram. Once the deployment finished, I disconnected the cable and I was free to move around with my hologram – a small race toy car. With a small tab in the air, I was able to move it around.

Space Recognition
HoloLens have space recognition- you see the space around so my car would fall from the edge of the coffee table and get stuck on impact with my backpack.

Building Apps
I started with a new project in Unity. Microsoft provided us with assets to use for the app. There are 3 main components/sensors you code for – gaze, tap and voice commands! We would add C# script for gaze, tab and voice recognition and attach them to objects.

Gaze
The little ring cursor will point at the object when your eyes/gaze reach the object.

Tap
The little tap with the index finger in the air will trigger a command.


Voice Recognition

I was able to implement in the code my personal commands. I was impressed how good the voice recognition was. It was easy to implement my commands for reset and drop the object. I think HoloLens actually interpret my accent better than the people around me.

The App
With the provided assets and code, my app had a small board with objects in it. There were two balls in the air that I was able to drop with a tap or a voice command. When the first ball dropped, an exposure would happen and the underground world would get exposed. I would place my gaze on the second ball and tap/give a command and the ball will drop in the underworld and I would be able to follow with my eyes the ball.

Interaction
I would be able to give my HoloLens to somebody else and they would experience my world. We were not able to interact with each other in one augmented reality. This functionality will be there in the future.

The Summary
When I saw the press release video from Microsoft in January, when I saw the demo at the keynote of //Build, my reaction was- no idea how this can be real. The experience with my hands on the HoloLens and the SDK surpassed my expectations by far! The augmented reality is very real, the HoloLens are comfortable and writing an app for HoloLens was not as hard as it sounds. I probably cannot explain the extent of my excitement and fascination. I have dreams now how to use the HoloLens, how to build apps for it. And after dreams and vision, now it’s the time to roll my sleeves and learn Unity…

Print Friendly

SoCal Code Camp

November 9 and 10, 2013 in Los Angeles

At the USC (University of Soutern California) campus in Downtown LA.
99 FREE 1-hour sessions
Around 500 developers.
$140K promissed in prizes but the prizes are really silly so if you go, go for the sessions.
It usually happens twice a year.

It’s a great free event but sometimes it’s hit and miss with the quality of the sessions.

Registration
So Cal Boot Camp

We enjoyed a great session on 3D gaming with Unity by a Microsoft Evangelist.
An intro to Unity for creating killer 3D Games
Adam Tuliper

So Cal Boot Camp

We went to a session on web hacking. It was a disasterous demo and the presenter stopped sharing his screen after 10 min. But he got just to talk for the rest 50 minutes. Neil is a Twitter security expert.
Intro to web hacking
Neil Matatall

So Cal Boot Camp

Learning everything about IIS in one hour. Steve gave everybody a free PluralSight 1-month promotion cards. I plan to use it.
IIS for Developers: Choose Your Own Adventure
Steve Evans

So Cal Boot Camp

We attended a web design session. Nice presentation but very slow and easy.
Getting Started with Responsive Web Design
Mario Hernandez

Website http://www.socalcodecamp.com/

Print Friendly

The //Build conference 2012

The Build conference is the biggest Microsoft developer conference. It’s famous for the fact that every attendee gets the newest and the coolest hardware. Last year it was the Samsung tablet (before being available on the market). This year it’s a 32GB Surface RT, 100GB of Sky Drive space and a Nokia Lumia 920 (not available on the market yet). Microsoft is giving developers an edge so they can bring the technology to live.

I was extremely lucky and very fortunate to attend the Build conference this year. It was in Redmond, Seattle on the campus of Microsoft. It sounds extremely cool and like a dream come true but the campus is just not made for an event this size. When you get to use the portable toilet before the keynote you kind of realize the problem.

The event had about 2,500 attendees and the online registration lasted only 50 minutes. The developers didn’t need convincing to pay $2,000 to attend the conference. The first 500 (the first 2min of the registration) got $500 discount. In my calculations with some of the expo sponsors that’s around $5,000,000 sold out in an hour without any marketing.

Even with all the popularity not everything is perfect in the World of //Build. The experience started with event registration. The Build conference barely has a website www.buildwindows.com . In August when the conference was announced the site consisted of one web form and that was it- no ‘about us’, no session catalog, no conference directory. We managed to register but 9 out of 10 registrants experienced website errors, including me.

The conference approached and we still didn’t have a real website. We received an email with bare bone schedule for the conference. Later the same was posted on the build website- it’s a one page website. I find it funny that we are talking about a developer conference and probably any one of us would’ve build the website or the app for free if we could’ve gotten a free ticket to Build :). I really wonder how many people are actually working on organizing the conference?

There were offers like a hackathon and a dinner with the Windows Phone team sent by email- any RSVP was done through an email, no website, no app. Not much is scalable- space was filling out very fast and if you saw the email 2h after it was sent, you were too late. Two days before the conference we still didn’t have a session catalog (I mean not one session listed). One day before the conference the session catalog was released on the channel 9 website. It was great they gave us a print out of the catalog when we were in line to register for the conference. I had more than enough time to choose my sessions when I waited 1.5h to register. Yup, the line for registration was ridiculous.

The sessions were great- truly honestly great and everybody can watch them for free on channel 9. I’ll be watching them myself (in the late evenings) as I spent most of my time at the hackathon. The channel 9 website is a true gift to developers. I should give credit to the organizers for posting the 150 videos pretty fast on channel 9. Great job!

The keynotes were awesome. It felt so good to have people like Steve Ballmer talking to us. It’s a real religion to most of us to come to the holy land and to spend a week between a crowd of brilliant people. Looking back, I would tell you this is probably the highest concentration of high IQ I have ever been in contact with. Unfortunately people were not thinking to network until the last session ended.

After the last session there was a phenomenon – people were starting a conversation everywhere and about anything. Unfortunately it was too late. Microsoft tried to make sure we are out of the campus as soon as possible. There was no hanging around the buildings after the end of the last session- you needed to be on your way.

When I have gone to other conferences the breakfast and lunch time has been the usual time to talk to new people. This time was different. The meal time was a pain. You had to walk to a big tent outside (it rained the whole week) and the food was not what I expected (Probably I’m spoiled from the TechEd food choices. The SharePoint conference meals are like a presidential gala compared to the Build meals). So after the first breakfast and portable toilet experience I kind of avoided the breakfast on campus- and ate breakfast at the hotel. The lunches I had to do but I never got to meet new people at lunch.

There was a Mixer on Monday night for the lucky people who got to finish their registration on time. I went and it was funny. I walk in and head for the different tables. In the sea of male faces I see a beautiful woman smiling at me. The biggest smile you can imagine- I smile back. We sit together and she tells me: I’m so happy you are here. There are no women and the guys don’t talk to me, they only talk amongst themselves. OK, that’s funny (especially because she is a very attractive woman and a speaker at the conference). So we have an interesting thing happening- maybe women not only get paid less but get send to conferences less as well.

The transportation… oh my. For the first time I go to a conference, I register for a hotel that’s on the conference website registration list and I need a rental car. Even at the conference party I had to take a taxi back from a different hotel. I don’t know why you list a hotel as a conference hotel if you don’t offer transportation to the hotel.

There were breaks of 45 minutes between the sessions which meant a lot of time wasted in transportation between building 33 and 92.

The popular sessions were not so easy to attend as well. Part of the problem of not having a website with the session catalog and people not building their schedules in advance is you don’t know which sessions are going to be the most popular. The most popular sessions were not in the biggest rooms. I had such a frustrating experience of being kicked out of session at the last minute because 20 of us were sitting on the stairs and the staff realized that’s a problem the moment the session started.

I didn’t go to the Beer Fest because I was at the hackathon. I heard I haven’t missed much and people were posting photos on Twitter of a deserted tent. The Beer really doesn’t make it a Fest- it’s the atmosphere.

The Conference party at the Armory… we spent long time on the bus to Seattle. I actually got to meet some interesting people (but I was an exception to the rule). Somehow developers don’t find it necessary to talk to the person next to them on the bus.

We arrived at our destination at the Armory. I’ve never seen so many bored people in one place. I know some people came and left after the first drink. It was a big tent looking building with probably 20 fast little bite food stands. The food ranged from junk to ridiculous with several exceptions (there was one sea food that was really nice). And like the whole week experience- for every bite you had to line up and wait sometime up to 10 minutes (for the sea food). I saw many people standing alone and staring at the ceiling. My advice is next time make it a hackathon event. This is concentration of brain power, excited brains that just learn new technologies and are itching to use them. Group them in teams- make them socialize and meet at least 5 new people. Make them brainstorm for some non-profit organizations and come up with brilliant ideas.

One interesting thing I noticed about Build is the high concentration of Europeans. I really love that. It was a truly international conference. The people I met were extremely bright and interesting. I regret not meeting more people. We were all so busy making the best of the knowledge base and learning.

The Expo! There was no expo floor. There were some desks around the session rooms in one of the buildings. Very small selection of sponsors were present and there was not much swag being given. I cannot believe this conference I came home only with one t-shirt!

No hands-on lab. Nothing Nada Zip Zero Zilch.

The Twitter #bldwin was probably my favorite. Build encourages every attendee and speaker to use Twitter. I love it. I met great people on Twitter #bldwin and some of them I met in person. Some of my favorite #bldwin people: ‏@ghowlett2020, @cwoodruff , ‏@noopman , @attilah , @TechMike2kX, @henriksen, @HammadRajjoub, ‏@aafvstam, @mangesnet, @kenstone, @samsabri.

On the last day of Build guess what? There was a Windows 8 and Windows Phone app released! Just in time for the last several sessions. This is truly funny. Duh… we were at THE conference for Windows 8 and Windows 8 Phone apps!

Would I do it again? Of course I would. My reasons? I got gadgets, I got to experience the technologies at the hackathon and I got to spend time around brilliant people.

What technology did I get excited about? Windows Azure mobile services and (Windows Phone 8) NFC.

Was it a good conference? Somehow Microsoft didn’t have any problem getting the people together- presenters and attendees but it looked as little organization went into it. Do you blame them? If it takes you 50 minutes to get 5 million dollars out of 2,500 people with one form website and no efforts to write even a page about the event- you know you can get away with anything.
***************************************************************
Other blog posts on //build/ 2012:
A blog post by Dennis Vroegop “It’s just badly organized, something I am not really used to in my 20 years of experience at Microsoft events.”
A blog post by Dennis Doomen “Whether or not this trip to Build 2012 in Redmond was worth the time and money”

Print Friendly

Participating in the Big Build Hackathon

If you ask me what will stay with me from the Build conference 2012 (Redmond / Seattle), I’ll tell you definitely the hackathon. I would give away my new Surface ( I got it as a present from the Build conference) just to do the hackathon again.

There are so many things that made this event memorable but I would like to start with a few that could’ve been better. The communication was so minimum that it hurt us. I wish I knew before I registered it’s not an 8 hour event but a 5 day event. I wish every participant at Build had a chance to participate. Like everything else at Build 2012 only the fastest and the luckiest got to register. After the registration there was an email with the schedule (3 days before the event). I couldn’t adjust my schedule on the late notice well enough- I missed the first two days because I had already planned my time (yup, the binder full of parties).

My biggest regret is we had no good way to network with people on different teams. I know we were all in the same room but we were all so observed in our own apps and teams. I wish we had our own little Mixer after the competition- I would’ve loved to talk to the other teams and ask questions about their apps. We were competitors the whole time and we should’ve had time to be friends. It was a wasted opportunity for us and for Microsoft to bring us together. I wish there was a tweeter group or something along those lines.

Food was served at the hackathon – all except lunch which was a problem. We wasted a long time walking back and forward to the outside tent with the attendee lunch. I didn’t understand why- it was just a waste of time. On Friday we actually didn’t have lunch because we wanted to see the demos- and by the time the demos ended the lunch tent was closed.

Even that things could’ve been better it was a brilliant event.

Let me tell you about the many things I loved of the event. We were encouraged to team up with people we never met before. I would’ve never believed that that could work but it worked like a charm. It was genius! Our team was originally five people but only three of us kept on going. We had to sacrifice the sessions and beer fest. At the start there were around 45 entries (from teams or individual participants) and most of them didn’t go to sessions, keynotes, parties and worked very long hours (sometimes the whole night). Almost every team had a participants from different countries.

We were given what felt as unlimited resources. There were at least 20 experts- the best of the best Microsoft employees ready to answer any question and guide us to accomplish anything we wished for. We were teamed up with mentors. Our mentor was Paul Batum- a truly impressive Azure mobile services expert. We were not limited to our main mentor- we talked with many of the mentors in the room. And there was email support for the people who worked remotely.

We all took advantage of Team Foundation Service ( tfs.visualstudio.com ). Teams of 5 or less can use TFS on the Cloud for free. It worked great- we all were checking in and checking out the pieces of the project we were working on. I love it!

Looking back I believe these were my mistakes- I didn’t sacrifice my first 2 days, I didn’t trust the team model at the start, I didn’t talk to many people(everybody was just coding) and we should’ve chosen to make a consumer app (not a business app).

There were no rules about what kind of app you should develop and no guidance was given but let me tell you- the apps that ended up being chosen were cool apps, ideas that you haven’t heard about before. We choose to develop too trivial app- a small business expense report system. Talking with other participants we came up with the conclusion That Microsoft is a bit tired of being seen as a business solution- they were looking to see cool consumer apps and these were the apps that were chosen to be demo’d. All apps that won used successfully Azure mobile services and the winning Windows Phone 8 apps used NFC.

There were 3 categories- Windows Azure, Windows Phone 8 and Windows 8. All apps needed to be started at the hackathon. The apps were officially judged on:
(34%) Innovative: Is this app meeting an existing need in a new way?
(33%) Applicability: Would this app be useful and appealing to a broad audience?
(33%) Technical Achievement: How difficult was it to build this app?

The prizes were not big (I have seen hackathons with prizes in the 6 digits). The total money won was $10K split between 9 teams (and teams were from 1 to 6 people). The biggest prize for the 1st place winners was the exposure. On Thursday everybody needed to submit their app by email to the first round of judges. 13 apps were chosen to be demoed in front of the celebrity judges. Celebrity judges included people like Scott Hanselman. The exposure to demo in front of the judges helped some people get job offers on the spot.

What we accomplished with our app was to implement the fast and fluid design of Windows 8. We had our screens, charms, contracts implemented. We had Microsoft account authentication (with the Live SDK). We used Azure mobile services and we were successfully writing data from our app to the cloud and displaying it back. We implemented the photo/file picker and the image upload to the cloud. We felt as we had a great app and a strong technical implementation. We didn’t even consider that we will be eliminated before the demo. The idea was reasonable- you submit your expense report (take a picture of the receipt and enter the information) and your boss receives and approves it. We were eliminated silently. We spent a lot of time, we sacrifice sessions and events, we accomplished a lot and I strongly believe we deserved the 3 minutes in front of judges.

The apps that won first place were:

Windows Azure category- the winner is QBranch. A team of 6 people from different countries developed a Windows 8 and Windows Phone 8 applications that used Windows Azure mobile services to store data. The app allowed you to have a digital queue for different events or places like DMV. The Windows phone 8 app reads NFC or QR codes to put users on the queue and uses push notifications to alert the user when their turn is up. It is developed in XAML and C#. The windows 8 app is used by the administrator to manage the queue. It is developed in HTML5.

Windows Phone 8 category – the winner is Social Squirrel. I felt this was the Microsoft favorite team and app- two people from Canada. Scott Hanselman offered them a job in the middle of their presentation. The rumor is that this team has already more than 10 apps in the Store. The app uses Facebook API and allows you to play a game and answer quizzes about your Facebook friends. The developers used the accelerometer to allow people to shake the phone and arrange the tiles randomly. NFC was used as well so two phones can communicate by touching. It is developed in XAML and C#.

Windows 8 category – the winner is Crowd Sourced Memories. A team of 4 attendees developed this cool app. At an event like a wedding the guests may use a Windows 8 machine to take pictures, videos and sign a message. All messages and photos are in the Cloud and can be played back in chronological order- even messages that were played back dot by dot as signed. The team mentioned they plan to have a windows phone 8 app and to implement NFC. It is developed in XAML and C#.

The winners have it all. They were interviewed for channel 9 and they truely deserve the fame and the prizes. What amazing work they did!

There is a big open question at the end of the hackathon for my team and most of the other teams. I believe Microsoft needs to help us with the next step. It would be such a waste if we don’t publish our apps to the store. We were encouraged to make teams. Most of the teams are international. How do we publish as a team? There are legal and accounting questions but the answer may be similar for each of our teams. I think this is an important last step that Microsoft should not overlook.

Dan Fernandez was the face of the Build Hackathon and he worked really hard. I suspect he barely slept for the duration of the hackathon. His efforts paid off. We all have memories to treasure and an experience to cherish. We learned a lot and we got inspired. I formed a team with two people that I just met and we created a strong bond. What a joy to see so many masterful developers and bright minds in one place! The experts in the room were phenomenal- extremely smart and knowledgeable Microsoft employees. I wish I had the names of all the mentors to give them credit. I’m so sorry I will miss your names and I only know your faces. Here are the names I know Paul Batum, Josh Holmes, Michael Johnson, Jeremy Foster.

Print Friendly