A good article by Bailey on how to leverage MCAS material to supplement classroom or home instruction.

Teachers and tutors can leverage any state standardized test material, or even National Assessment of Educational Progress (NAEP) material. After each assessment, NAEP releases dozens of sample questions to the public—more than 2,000 questions are currently available to be used in classrooms, as homework, or for assessments. These questions are of high quality and additional details are tagged to each item, such as subject classification, difficulty, and performance on the question by region and various demographics.

While working with teachers we repeatedly hear how they print the questions from these websites; then they go through a tedious process of cut-and-paste to create custom assessments and worksheets for specific subject areas. This inspired us to create the QuestionCart tool in Socrato.  The QuestionCart allows teachers (and anyone else) to search questions for MCAS and other exams by grade, question type, state standards, and other options.  Teachers can then add the questions to a QuestionCart similar to a shopping cart experience on other websites.  Once the question cart is populated, a teacher can click a button to convert those questions into an assessment test or printable worksheet with one click. No more cut and paste!

An open letter to the Massachusetts Commissioner of Elementary and Secondary Education:

Mitchell D. Chester, Ed.D.
Commissioner of Elementary and Secondary Education

Dear Commissioner Chester,

I work with a number of schools helping them review MCAS data and results. Thank you for shortening the turnaround time for MCAS results for schools, and for shortening the test periods.  Schools will greatly benefit from the increased analysis and planning window you have created.  However, the new policy regarding the release of only a partial set of questions greatly impacts the ability of teachers and schools to review those MCAS results in a meaningful way.

Having a report showing only strand information puts teachers at a big disadvantage. It is very difficult without the actual questions to see where students are having difficulty.  For example, the strand information does not indicate whether a math question is a whisker plot, a line graph, or a stem and leaf plot; students may have done well on one but not another.

ELA questions are even more problematic. The strand indicators are almost meaningless (fiction, non-fiction, poetry, etc.) because they don’t tie back to student skills, such as “finding evidence”, or “making inferences” about the passages.  When the questions and answers are released and provided to the teachers, these interpretations can be made easily; without the questions, it is almost not worth looking at the English MCAS results.  While it is not necessary to release all the reading passages, the questions and answers are fundamental to proper analysis.

Being able to see the question and answer content also allows teachers to see which answers the students indicated, showing the answer response patterns of their classes.  Many times students choose the same wrong answer, showing that they misunderstand a concept in the same way.  For example, in math when calculating an expression with absolute value operations and squaring, a number of students might calculate the square correctly, but not get the proper sign.  Again, without access to the questions, this type of inference is impossible.

Release of test items is critical for teachers to conduct a meaningful analysis, and to help their students make real progress.  While it may allow for shortening the test period, and perhaps save money in a time of state budget scarcity, it is a big step backward for MCAS.

I am sure you have considered other options, but I implore you to look more carefully.  I am confident there are a set schools that would allow future questions to be field tested, perhaps in an interim assessment in November.  Such an approach could provide a win-win situation.  Schools who participate in the interim assessment program would get better information about student progress before the spring exams, and the entire community would benefit from the release of the common test items from the March and May assessments.

I urge you to support schools and teachers by reversing the new policy that limits the release of test items.

Warm regards,

Mike Oates

mfoates@gmail.com

How Data analysis is helping raise performance – story covers both good and the bad
Data-Driven Schools See Rising Scores - WSJ.com

Educational data is in abundance – but not in very useful state.
Education Week: Data, Data Everywhere, But Not a Drop to Use 

Impact of providing detailed and real time access to student educational and learning data to parent.
Parents Will Get Access to Student-Tracking Site - NYTimes.com

How internet based solution are taking the education to next level. With easy assess to past exam and solutions, teachers and professors need to constantly iterate and improve their material.

Psst! Need the Answer to No. 7? Click Here. - NYTimes.com

Inspiring presentation and definition on “Definition of Success” by Richard St. John reminding, that success is not a one-way street, but a constant journey. He uses the story of his business’ rise and fall to illustrate a valuable lesson — when we stop trying, we fail.

If you get chance check it out at TED website.  

Enjoy!

NYtimes posted a story on new charter school. It’s inspiring to see that Mr. Zeke M. Vanderhoek founder of a test preparation company Manhattan GMAT is assembling a dream team of teachers to kick start a charter school.

I agree with the approach that excellent and passionate teachers / tutors are the key ingredients to success. Seems, the money is not an issue and this charter school can afford to pay up to $125,000/year to teachers. I believe that Mr. Vanderhoek background as an entrepreneur, experience in tutoring, and a dream team will generate a positive outcome for this experiment.

The question is, with financial, union and political challenges out there, can this experiment be scaled easily?

For people who do not believe that private tutoring can help increase SAT scores, this article about South Korean tutoring industry is intense. A recent study by NACAC got coverage in WSJ that private tutoring does not help much in raising SAT scores. At least, that’s how the finding of the report is being projected. If you dig deeper in article and report, it says lot more research need to be conducted.

In South Korea tutoring companies are traded on stock exchange. Teacher and tutors are earnings millions of dollar per year. Tutors are considered as pop stars. Similar is the case in other Asian countries. Because of hyper competitive environment, high cost of higher education and potential for better life, parents want to provide every advantage they can to their kids, and one of the solutions is tutoring.

With competition and cost of higher education growing in US, is this a sign of what going to happen in US in few years?

Today we upgraded Socrato with new release.

Major updates were done in learning analytic and reporting area. Key high lights include:

• Teachers or tutors can create custom reports for group of students.
• Assessment data can be analyzed by teacher defined parameters and criteria. Example, school staff can look at the performance of a class by race, gender, lunch code or any other parameters they wish to define. This is already providing interesting insights to our current users.
• Learning data analytics is very tightly connected with learning content. Teacher can assemble relevant learning content just by clicking on the standards link in various reports.
• Question distribution report now shows the list of students for each response. 

In addition, number of bug fixes were made and usability issue where resolved.

As always, thank you all for your feedback. If you run into any issues please drop us an email or leave a comment.

With the MCAS season underway and teachers trying to help students, we received a few requests to get insight into how the questions are distributed across the learning standards. Or, how many questions are allocated to each standard area in a given year or grade level? Or, Do you have the frequency analysis of questions that has ever been given in the past few years of the MCAS exam?

As we started working on one off requests, we decided to put the entire material in a simple to use spreadsheet so it can benefit as many teachers and tutors as possible. To use the spreadsheet and see this for yourself click on the MCAS Question and Content Analysis-v2.xls in your browser. In order for this to work, you will need Microsoft Excel (part of MS Office suite).

Save the spreadsheet on your local drive, open the spreadsheet and click on the “MCAS-TestAnalysis” Tab.

On this tab make the selection for various choices in the drop down boxes on the upper left:

• Grade level  (Row 3 )
• Year published (Row 4)
• Test Name (Row 5)

You can further refine the data on Row 8 by filtering

• Subject name
• Topics name
• Standard name

By default the report shows data for grade 6 but you can change that in Row 3 by making other selection.

The question distribution report will show as you make selection changes. I hope you will find this useful. Few teachers are using this data to create a series of formative assessments that focus on the highest frequency questions for each strand.

If you have any feedback or comments please let us know. Also, feel free to pass this along to any other teachers who you think will find it useful.

In the future we will try to hook this report into Socrato as a feature so you will have access without Excel. For more interesting content and learning data analysis checkout www.socrato.com or contact us to setup a demo.

Best Regards,
-Raju

UPDATE - uploaded the new version of excel file - this has the full description of each standard.

Many industries use business intelligence software tools to convert their data into information, and leverage that information to make business decisions. It’s only a matter of time before the same tools, technology and processes become main-stream in the educational space.

I came across this article in School CIO; it’s about the use of a business intelligence in education. It’s encouraging to see these examples as tools and technology like SAS and Cognos find their place in education. These are very powerful tools educators and school systems can benefit from. These tools still need to be simplified to increase the adoption. Education has lot to gain with this convergence. If you get chance, check out the article. Let me know what your opinion is, and how you are using the intelligence technology in your environment.

-Raju

We are pleased to announce that this past weekend we rolled out a new release on Socrato. Some of the changes are external and can be seen by users, but most of the improvements are “under the hood” and result in improved response time. The key changes include:

User Interface

• More content  - In addition to grade 3 – 5 MCAS material, more content was added for grades 6 and 7.
• Easier access to content - Question search by grade level.
• Improved reporting - The Question distribution report now shows students’ points-data for open response questions.
• More stable upload of students’ assessment data.
• Fixed a number of issues for students TIA and roster data upload process. (e.g., information for one question was captured incorrectly and the session ID was getting lost).
• Fixed a number of other bugs.

Under the Hood

• Faster performance for take test area - With a number of schools now using Scorato in their school computer labs, we needed to make improvements so the system does not slow down. We made a good number of changes to how the take test is managed in the system. This will allow our servers to support a larger number of concurrent users.
• More stable and faster reports generation – We have completely revamped how the PDF files are generated for SAT diagnostic reports. With this change, the time required for report generation has dropped significantly. The current load calculation shows that the first report is available in about 10 -15 seconds, and subsequent reports are created in less than 5 seconds. These are complex reports, so this improvement constitutes a significant improvement compared the previous time lapse required to generate the reports. Pretty cool!

If you encounter any issues with these changes and improvements, please let us know.

Looking Forward

New and improved key features which are currently works in progress include:

• Additional reports which will allow teachers and principals to slice and dice their school’s and students’ data. They will be able to leverage these reports for instruction planning. This will allow them to automatically create custom and personalized assignments from Scorato. We are very excited about this.
• Making the delivery (printing and emailing) of all reports easier.

A big thank you goes out to all who have sent suggestions, requests and issues to us. We welcome your feedback and try to act on it in a timely fashion.

-Raju