main
members
work
about me
avatar
Paul(admin)

K-nearest-neighbor algorithm

Post replyCan't edit post
Watch topicCan't delete post
Thursday, March 25 2004
Download the document below.
K-nearest-neighbor is a popular method of solving problems in computational intelligence. The report explains the Visual Basic application of the algorithm in further detail.
The application, source code and summary report are in English.
Update 16th July, 2007: A JAVA version of this code is now also available from this page.
Update 25th October, 2007: I updated the application to version 1.1, which now supports Forward Selection. More importantly, however, is that I translated the summary report into English and significantly improved it, clarifying how the algorithm works.
Download
knn.pdf (648.45 KB)
knn.zip (51.27 KB)
Application programmed in Visual Basic 6.0 SP1
Summary report written in Microsoft Word 2007
Rating:
Actions:Post replyWatch topic
avatar
Paul (guest)
comment
Saturday, November 25 2006, 14:43
Can't edit commentCan't delete comment
Thank you
avatar
Thank you very much (guest)
comment
Friday, January 12 2007, 9:03
Can't edit commentCan't delete comment
Dear Paul,
Thank you very much.
May I get knn code in JAVA?
May email address: [removed]
Thanks & regards,
Rita
avatar
Paul (admin)
comment
Friday, January 12 2007, 12:56
Can't edit commentCan't delete comment
Thank you for your feedback, Rita. I've sent you an e-mail with more information.
Also, I removed your e-mail address above to prevent any abuse (spam or otherwise).
avatar
Rebecca (guest)
comment
Monday, January 15 2007, 20:00
Can't edit commentCan't delete comment
Paul - thank you very much for providing this example - it was very well done and informative.
avatar
Ali (guest)
comment
Sunday, January 28 2007, 22:23
Can't edit commentCan't delete comment
Hi Paul,
I was wondering if the KNN code you have, could it work for images. I need KNN for image processing. Also, if i can get the code in C/C++, that'd be great.
Regards,
Ali.
avatar
Paul (admin)
comment
Monday, January 29 2007, 15:03
Can't edit commentCan't delete comment
Boy, if it'd work with images! No, I'm afraid it can't do that.
If you could manage to find an effective way of determining a qualifying factor for a target image, say x factors, you could run the algorithm on a test set and evaluate the quality of each factor using cross-validation (for example the relatively simple leave-one-out-cross-validation).
Better yet would be if you could write an algorithm that randomly enumerates an x number of factors and decides on its own which ones give the best results.
Some rules that come to mind would be measuring the color/luminance of a pixel (or set of n×n pixels), the distribution over a row or column, overall hue, etc. etc.
I'm afraid I don't have a C/C++ version of this code. Visual Basic is an extremely accessible programming language, it should be relatively easy for a C-programmer to understand. I thought I had a Java version somewhere, but I can't find it at the moment.
avatar
Bawya.S (guest)
comment
Friday, March 23 2007, 6:52
Can't edit commentCan't delete comment
hai paul,
Do you have the code in C if you have can you please send me.
looking for reply eagerly.
With regards,
Bawya.S
avatar
Paul (admin)
comment
Friday, March 23 2007, 15:16
Can't edit commentCan't delete comment
No, I'm afraid I don't have a version with C code. Like I said before, Visual Basic is rather easy to understand and you might want to try translating it to C yourself.
Also, it was brought to my attention that the download files were broken; this has been fixed and improved.
avatar
Jas (guest)
comment
Tuesday, June 26 2007, 11:01
Can't edit commentCan't delete comment
hi paul,
may I get knn code in java?
my email address: jasmine [underscore] siy [at] yahoo [dot] com
thans a lot
God bless
jas
avatar
Neeba N.V (guest)
comment
Tuesday, July 10 2007, 19:48
Can't edit commentCan't delete comment
Hi Paul,
Can I get a linux version of the code.
avatar
Paul (admin)
comment
Wednesday, July 11 2007, 15:50
Can't edit commentCan't delete comment
I'm afraid not. Visual Basic is only for Windows, and presently I don't have a Java (or C++) version. I've gotten so many requests over the years that I'll consider writing one, but no guarantees.
avatar
Paul (admin)
comment
Monday, July 16 2007, 1:16
Can't edit commentCan't delete comment
There has been a lot of interest in a Java version of this code. I am pleased to announce that I have released such a version, which is now available for download from this page. Feedback is, as always, welcome!
avatar
Andrea (guest)
comment
Wednesday, October 24 2007, 12:20
Can't edit commentCan't delete comment
Hi Paul
I'm interested in your implementation. Do you have a English version of a document?
Thank you
Andrea from Brazil
avatar
Paul (admin)
comment
Thursday, October 25 2007, 16:23
Can't edit commentCan't delete comment
You're in luck, Andrea! Typically I'm too busy to do that kind of thing, but I had been wanting to get around to that, so I updated both the application and the summary report.
The report is now in English and has been substantially expanded and clarified. I hope this helps you.
avatar
Igor Stjepanovic (guest)
comment
Thursday, November 22 2007, 8:25
Can't edit commentCan't delete comment
Hi Paul,
is your implementation of KNN capable of performaing spatial searches? I have a large list of points (250,000) in lattitude/longitude and for a given point I need to find 1-5 nearest points. The given point will not be in the list.
Is this possible with your KNN?
Thanks,
Igor
:)
avatar
Paul (admin)
comment
Thursday, November 22 2007, 19:27
Can't edit commentCan't delete comment
No, this implementation is not suited your proposed problem.
With some code modifications you might be able to fix it up to do that, but I wouldn't recommend this particular application as it only supports a small dataset and only understands classes with non-scalars.
Instead, you might find it worthwhile to look at my Java implementation, jKNN, which is less strict. Also, the code is more understandable and easier to adapt.
Either way, a lot of tampering with the code will be required in order to get it to do what you want.
Good luck!
avatar
Jack (guest)
comment
Tuesday, April 22 2008, 6:38
Can't edit commentCan't delete comment
Dear Paul,
Thank you very much.
May I get knn code in JAVA?
May email address: [yihanzd@163.com]
Thanks & regards,
Jack
avatar
Paul (admin)
comment
Tuesday, April 22 2008, 12:48
Can't edit commentCan't delete comment
In both the original post and in my previous comment I've placed a link to the Java version.
avatar
Myriam Arias (guest)
comment
Tuesday, April 29 2008, 10:34
Can't edit commentCan't delete comment
Hi Paul,
Sorry if you got this twice but I really need your help. Thank you for providing the application and summary report. It was very well done. But I have a problem, how can I load the dataSetSituations from a string array for example. I tried but i got error "cannot convert from String to KnnValue" and "cannot cast from a String to KnnValue". Can you give some advise please.
Best regards,
Myriam
avatar
smartrohitkumar (member)
comment
Wednesday, April 30 2008, 8:17
Can't edit commentCan't delete comment
Hi
Myself Rohit Kumar Singhal :(
I want Knn implementation for Mobile Customer Relationship Database
Please Help Me
Bye
avatar
harish118@hotmail.com (guest)
comment
Tuesday, October 28 2008, 4:06
Can't edit commentCan't delete comment
hi,
Just curious if I can use the same method to find where along a given set (a super set) of (X1,Y1....Xn,Yn) Coordinates does another set (Sub Set) of Points (A1,B1...Am,Bm) align? Or are at the closest.
Thanks a lot & Regards,
Harish
avatar
Paul (admin)
comment
Sunday, November 2 2008, 3:32
Can't edit commentCan't delete comment
It would seem to me (if I understand your question correctly) that you are simply looking for a sorting algorithm, Harish. If you want to determine which points are nearest to each other, simply compute the Euclidean distance from each point from the first set to each point in the second set.
I would suggest you look in to the Hausdorff distance or the Mahalanobis distance, depending on whether one of the sample sets is unknown or not (i.e. taking probabilities into account or not).
avatar
Paul (admin)
comment
Tuesday, November 11 2008, 4:37
Can't edit commentCan't delete comment
An alert commenter notified me of a minor bug in the Visual Basic version of the code, that caused training set examples with distances of precisely 0 not to be registerd as a nearest neighbor. I've fixed the problem and updated the code provided via the link above.
avatar
Paul (admin)
comment
Tuesday, November 11 2008, 23:39
Can't edit commentCan't delete comment
I've made some additional modifications to the application, including improved support for Vista. The foremost change in functionality is that in scaling mode, the test scenario is now scaled as well, opposed to only scaling the situations training set.
The new version, 1.2, is now online and available for download via the above link.
avatar
muhendis (guest)
comment
Monday, May 18 2009, 18:22
Can't edit commentCan't delete comment
Hello! thanks for information about this subject and article..
But i have a problem that your winrar archive for KNN algortihm in VB or JAVA gives errors when extracting, please help me to see your full codes may you please send me knn codes in java and vb?
thanks a lot.
avatar
Paul (admin)
comment
Tuesday, May 19 2009, 1:38
Can't edit commentCan't delete comment
A bug in Internet Explorer was causing the downloaded files to corrupt; any other browser would have worked. I patched it up for IE, though; the download should be working fine again.
avatar
muhendis (member)
comment
Tuesday, May 19 2009, 7:27
Can't edit commentCan't delete comment
Thank you so much, i have downloaded the rar files now without any problem. Thanks for your attention..
avatar
ek20a (guest)
comment
Sunday, October 11 2009, 22:56
Can't edit commentCan't delete comment
Hi, I think that you've made mistake writing formula for euclidean distance number 2.
http://en.wikipedia.org/wiki/Distance
avatar
Paul (admin)
comment
Monday, October 12 2009, 2:27
Can't edit commentCan't delete comment
Yes, you're right: it presently states Euclidean distance as:
∑ ( √ ( x² - y² ) )
This should be:
√ ( ∑ |x - y|² )
avatar
tamiyc11 (guest)
comment
Friday, May 5 2017, 14:20
Can't edit commentCan't delete comment
Pron blog site
http://asian.adultnet.in/?nikki
erotic france good erotic movies erotic painting erotic shows erotic nights
Post reply Hide comments
Please contact me prior to distributing any files.
Members have extra privileges. You can login or register.
© 2005–2017 P. F. Lammertsma
No members currently online; 541 hits today by 181 unique users