Using Ajax Calls in Django Framework

Django is a high level Python based web framework. In order to create a working project in Django, the basic components includes a views which is a python file with the backend logic, an html template that would render all static components and a urls file that links these two python and html files. But what if we want to add some dynamic content to our html page or update the page on fly. We don’t want to refresh it each time we receive a new input from user or process it. For this purpose we use Ajax which uses XMLHTTPRequest to update content of the existing html page without refreshing it. Lets take a scenario where Ajax is used within Django framework to dynamically update the contents of the page. For example, say our html template loads the first time with a div that has a form to enter number of records to retrieve from database. Based on the user input, we display the contents using ajax and not reloading the entire page.

The initial base.html would render all static content and include all necessary jquery and css files and this would be inherited within our specific html template say myreport.html. This myreport.html would look something like this.

# extends 'base.html'
{% block content %}
<!-- This block is loaded initially to ask for user input -->
<div class="row-fluid">
	<div class="span12">
		<div class="widget-box">
			<div class="widget-title">
				<span class="icon">
					<i class="icon-list-alt"></i>
				</span>
				<h5>Input Values from User</h5>
			</div>	
            <div class="widget-content">
                Enter Number Of Records: <input type='text' name='numrecs' id='num_recs'>
                <button type="submit" value="View Report" onclick="send_request()>View Report</button>
            </div>
		</div>
    </div>
</div>
<!-- This block is loaded only after ajax call retrieves the data from table -->
<div class="row-fluid" id='example_table' style="display:none;">
	<div class="span12">
		<div class="widget-box">
			<div class="widget-title">
				<span class="icon">
					<i class="icon-list-alt"></i>
				</span>
				<h5>Table Name</h5>
			</div>	
            <div class="widget-content"></div>
			<div id="placeholder"></div>
    	</div>
	</div>
</div>
{% endblock content %}	

{% block script %}
<script id="source" language="javascript" type="text/javascript">
//This sends the user input on number of recs to views
function delete_request()
{
    params = {'records':$("#num_recs").val()};
    var serializedData = jQuery.param( params );
	$('#info_form').remove()
	$.ajax({
	  type: "POST",
	  url: "{{ lib.url('json_function') }}",
	  data: serializedData,
	  success: function(return_val){show_table(return_val[0]);},
	  dataType: 'JSON'
	});
}
//renders the report
function show_table(data)
{			
	var str="";
	var i=0;
	var j=0;
	if(data.hasOwnProperty('report'))
	{
		str="<table class='table' id='example_table'><thead><tr>";
		for(i=1;i<data.headers.length;i++)
			str=str+"<th>"+data.headers[i]+"</th>";
		str=str+"</tr></thead>";
		str=str+"<tbody><tr>";
		for(i=0;i<data.report.length;i++)
		{
			var row=jQuery.parseJSON(data.report[i]);
			for(j=1;j<data.fields.length;j++)
			{
				field=data.fields[j];
				str=str+"<td>"+row[field]+"</td>";
			}
		}
		str=str+"</tr></tbody>";
		str=str+"</table>";
		$('#placeholder').html(str);
		$('#example_table').css('display','block');
	}
	//table initialization
	var oTable = $('#example_table').dataTable( {
	    "bJQueryUI": true,
	    "bRetrieve": true,
	    "sDom": '<"F"lfT>t<"F"p>',
	    "iDisplayLength": 10,
	    "sPaginationType": "full_numbers",
	    "aaSorting": [[ 0, "desc" ]],
	    "oTableTools": {
			"sSwfPath": "/static/plugins/TableTools/media/swf/copy_csv_xls.swf",
			"aButtons": [
				"print",
				{
					"sExtends":    "collection",
					"sButtonText": 'Export <span class="caret" />',
					"aButtons":    [ "csv", "xls", "pdf" ]
				}
			]
		}
	} );  
}
</script>
{% endblock script %}

Next within our views.py we will create the function to render this html template and the function that will handle the ajax requests.

'''This page returns nothing except rendering the static form for user input'''
@render('myreport.html')
def myreport(request):
    return {
    }
'''This handles the ajax request'''
def json_function(request):
	data={}
	jdata=[]
	report = []
	records_to_retrieve = 0
	url = request.REQUEST
	fields = ['fieldA','fieldB','fieldC','fieldD']
	headers = ['A','B','C','D']
	if 'info_request' in url and 'records' in url:
		records_to_retrieve = int(url['records'])
		temp = DatabaseModel.objects.all().values(*fields)[:records_to_retrieve]
	try:
		records = CapacityDemand.objects.get(id = int(url['demand_id']))
		for row in records:
			report.append(json.dumps(row)) 
	data['report']=report
	data['headers']=headers
	data['fields']=fields
	jdata.append(data)
	response = json.dumps(jdata)
	return HttpResponse(response, mimetype="text/plain")

And in the end we link these two files in the urls.py file as follows:

    url(r'^myreport/$', 'appname.views.myreport', name='myreport'),
    url(r'^json_function/$', 'appname.views.json_function', name='json_function'),

Service-Now data pull – python vs perl

Service-now is a IT service management software that hosts a wide ranging list of reports from different modules of IT services that are organized into tables. ServiceNow publishes its underlying table structures and associated data that can be pulled via SOAP query. This can be achieved through any scripting language. Although, python is my default goto scripting language, the soap client in python are not very well maintained. The most pythonic of those is the SUDS which does not seem to work well with service now api. While it is pretty slick and easy to get the basic data output using python/suds, as you move to more complex queries python fails. This is a known issue.

https://fedorahosted.org/suds/ticket/330
http://lists.fedoraproject.org/pipermail/suds/2011-October/001526.html
http://stackoverflow.com/questions/11850275/parameters-with-leading-underscores-in-python-suds

The __limit and encoded query does not seem to have the desired effect on the output as SUDS never passes these. So after few trials I ended up using Perl for the data pull and calling Perl from my Django framework. This worked like a charm. So here goes a snippet that can be used to get multiple records from an incident with max limit 10000 records and pulling single records from incident.

#!/usr/bin/perl -w
use SOAP::Lite;

# basic auth
sub SOAP::Transport::HTTP::Client::get_basic_credentials {
return 'user' => 'password';
}

# specify the endpoint to connect. This is the first incident that pulls say two values field1, field2 and field3 is used as a filter for records.
my $soap1 = SOAP::Lite -> proxy('https://instance.service-now.com/incident1.do?displayvalue=all&SOAP');
my $method1 = SOAP::Data->name("getRecords")-> attr({xmlns => "http://www.service-now.com/"});
my @params1 = ( SOAP::Data->name(field3=> "Your Value") );
push(@params1, SOAP::Data->name(__limit => "10000") );
my $som1 = $soap1->call($method1 => @params1);
my @serverData1 = @{$som1->body->{getRecordsResponse}->{getRecordsResult}};
foreach my $serverRec (@serverData1) {
	$f1 = $serverRec->{field1};
	$f2 = $serverRec->{field2};
	print $f1,',',$f2,"\n";
}
#incident 2 match  field4 and then retrieve field5 and 6 for single record
my $soap2 = SOAP::Lite -> proxy('https://instance.service-now.com/incident2.do?displayvalue=all&SOAP');
my @params2 = ( SOAP::Data->name(field4 => "Your Value") );
push(@params2, SOAP::Data->name(__limit => "1") );
$f5 = $soap2->call($method1 => @params2)->body->{getRecordsResponse}->{getRecordsResult}->{field5} || "None";
$f6 = $soap2->call($method1 => @params2)->body->{getRecordsResponse}->{getRecordsResult}->{field6} || "None";
print  $f5,',',$f6,"\n";

And then from python function we can easily call this perl script:

import subprocess
ret = subprocess.call(["perl","/path/name/only/servicenow.pl"])